aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Pages: 1 2 [3]

Author Topic: High CPU usage during procedure of final file hashing  (Read 15660 times)

GonoszTopi

  • The current man in charge of most things.
  • Administrator
  • Hero Member
  • *****
  • Karma: 169
  • Offline Offline
  • Posts: 2685
Re: High CPU usage during procedure of final file hashing
« Reply #30 on: March 13, 2009, 09:30:29 AM »

Or you can try
Code: [Select]
echo x | cpp -dM - | sort to see all predefined symbols.
Logged
concordia cum veritate

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #31 on: March 13, 2009, 02:40:32 PM »

Stu Redman
can't find __powerpc__
only
Code: [Select]
#define __i386__ 1
#define __tune_i486__ 1
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: High CPU usage during procedure of final file hashing
« Reply #32 on: March 14, 2009, 09:54:01 PM »

Do amule try to perform aligned memory access with powerpc or don't care about it?
And why powerpc is not mentioned in ArchSpecific.h?
Now - do you have a PowerPC build environment (which can't have __i386__ defined) or was your question purely academical?
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #33 on: March 14, 2009, 11:11:50 PM »

Stu Redman
My question wasn't academical, it's practical.
I suppose that my building environment is hardcoded into crosstoolchains.

I'm trying to look for a cause of  that inefficiency:
Code: [Select]
final file hashing speed:
rtorrent + openssl     ~150 MB/minute  with    NO   system slow down
amuled + cryptopp     ~50 MB/minute    with    FULL   system slow down
Everything was built with  CFLAGS="-O3 -mcpu=603e -mtune=603e" CXXFLAGS="-O3 -mcpu=603e -mtune=603e"

So without knowledge of programming I can only guess about the real cause - may be amuled or/and cryptopp make a monkey work because of wrong alignment/endianness or I don't know what else it could be...
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: High CPU usage during procedure of final file hashing
« Reply #34 on: March 14, 2009, 11:34:37 PM »

So if you have a PowerPC cross compile tool chain, please look for the predefines in it (and not in your machines native compiler like you obviously did last time if __i386__ was defined).
I can't add support for PowerPC if I don't know the predefine. I'm quite sure it's __powerpc__ , but I'd like to have it verified by you.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #35 on: March 14, 2009, 11:53:34 PM »

There is no predefined __powerpc__ at all, neither in native nor cross gcc.

In "powerpc-linux-uclibc-gcc -dumpspecs" output I find  -D__powerpc__ but it's only for BSD, not Linux.
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: High CPU usage during procedure of final file hashing
« Reply #36 on: March 14, 2009, 11:59:04 PM »

Or you can try
Code: [Select]
echo x | cpp -dM - | sort to see all predefined symbols.
Please do just that with your powerpc-linux-uclibc-gcc.  ::)
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #37 on: March 16, 2009, 10:33:49 PM »

Strange, but I get different hashing  speed for just downloaded files and for added by hand to Incomining directory  files:
- added file                 ~115 MB/minute  with slight   system slow down
- downloaded file     ~50 MB/minute   with  FULL   system slow down
Why?
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: High CPU usage during procedure of final file hashing
« Reply #38 on: March 16, 2009, 10:44:17 PM »

If you start issues and then don't answer my questions I see no reason to help you.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #39 on: March 19, 2009, 08:28:51 PM »

The problem has disappeared by itself after downgrade CryptoPP version 5.5.2 to 5.2.0
Now the speed of hashing is ranging from 30 MB/min up to 120 MB/min, depending on if rtorrent is active and if active how busy it(rtorrent) is and WITHOUT any system slowdowns.

BTW, I also carried out an experiment with new CryptoPP version 5.6, but the result was even worse  - the hashing thread of aMuled used CPU at 80%...90% with full system slow downs.

Why the hashing thread of aMuled with newer versions of CryptoPP tried to act as real-time process  not letting any other processes and even the system to use CPU in concurrent manner remains for me to complete mystery.


Stu Redman
There is NO  predefined __powerpc__ for Linux target platforms in my crosstoolchains, it is predefined only for BSD.

freddy77
Thanks for mmap patch.
While it does not affect the CPU load in my case, but it is useful for memory usage - now aMuled try to use memory in wave-like manner.
Logged

phoenix

  • Evil respawning bird from aMule Dev Team
  • Developer
  • Hero Member
  • *****
  • Karma: 44
  • Offline Offline
  • Posts: 2503
  • The last shadow you'll ever see
Re: High CPU usage during procedure of final file hashing
« Reply #40 on: March 23, 2009, 01:51:29 PM »

p.r,

Out of curiosity, could you post the results of  "gcc -dM -E - < /dev/null | sort"?

Cheers!
Logged

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #41 on: March 23, 2009, 07:34:30 PM »

phoenix
Code: [Select]
#define PPC 1
#define _ARCH_PPC 1
#define _BIG_ENDIAN 1
#define _CALL_SYSV 1
#define __BIG_ENDIAN__ 1
#define __CHAR_BIT__ 8
#define __CHAR_UNSIGNED__ 1
#define __DBL_DENORM_MIN__ 4.9406564584124654e-324
#define __DBL_DIG__ 15
#define __DBL_EPSILON__ 2.2204460492503131e-16
#define __DBL_MANT_DIG__ 53
#define __DBL_MAX_10_EXP__ 308
#define __DBL_MAX_EXP__ 1024
#define __DBL_MAX__ 1.7976931348623157e+308
#define __DBL_MIN_10_EXP__ (-307)
#define __DBL_MIN_EXP__ (-1021)
#define __DBL_MIN__ 2.2250738585072014e-308
#define __DECIMAL_DIG__ 17
#define __ELF__ 1
#define __FINITE_MATH_ONLY__ 0
#define __FLT_DENORM_MIN__ 1.40129846e-45F
#define __FLT_DIG__ 6
#define __FLT_EPSILON__ 1.19209290e-7F
#define __FLT_EVAL_METHOD__ 0
#define __FLT_MANT_DIG__ 24
#define __FLT_MAX_10_EXP__ 38
#define __FLT_MAX_EXP__ 128
#define __FLT_MAX__ 3.40282347e+38F
#define __FLT_MIN_10_EXP__ (-37)
#define __FLT_MIN_EXP__ (-125)
#define __FLT_MIN__ 1.17549435e-38F
#define __FLT_RADIX__ 2
#define __GNUC_MINOR__ 3
#define __GNUC_PATCHLEVEL__ 3
#define __GNUC__ 3
#define __GXX_ABI_VERSION 102
#define __INT_MAX__ 2147483647
#define __LDBL_DENORM_MIN__ 4.9406564584124654e-324L
#define __LDBL_DIG__ 15
#define __LDBL_EPSILON__ 2.2204460492503131e-16L
#define __LDBL_MANT_DIG__ 53
#define __LDBL_MAX_10_EXP__ 308
#define __LDBL_MAX_EXP__ 1024
#define __LDBL_MAX__ 1.7976931348623157e+308L
#define __LDBL_MIN_10_EXP__ (-307)
#define __LDBL_MIN_EXP__ (-1021)
#define __LDBL_MIN__ 2.2250738585072014e-308L
#define __LONG_LONG_MAX__ 9223372036854775807LL
#define __LONG_MAX__ 2147483647L
#define __NO_INLINE__ 1
#define __PPC 1
#define __PPC__ 1
#define __PTRDIFF_TYPE__ int
#define __REGISTER_PREFIX__
#define __SCHAR_MAX__ 127
#define __SHRT_MAX__ 32767
#define __SIZE_TYPE__ unsigned int
#define __STDC_HOSTED__ 1
#define __USER_LABEL_PREFIX__
#define __USING_SJLJ_EXCEPTIONS__ 1
#define __VERSION__ "3.3.3"
#define __WCHAR_MAX__ 2147483647
#define __WCHAR_TYPE__ long int
#define __WINT_TYPE__ unsigned int
#define __gnu_linux__ 1
#define __linux 1
#define __linux__ 1
#define __powerpc 1
#define __powerpc__ 1
#define __unix 1
#define __unix__ 1
#define linux 1
#define powerpc 1
#define unix 1
Logged

GonoszTopi

  • The current man in charge of most things.
  • Administrator
  • Hero Member
  • *****
  • Karma: 169
  • Offline Offline
  • Posts: 2685
Re: High CPU usage during procedure of final file hashing
« Reply #42 on: March 23, 2009, 08:50:11 PM »

There is no predefined __powerpc__ at all, neither in native nor cross gcc.
Code: [Select]
#define __powerpc__ 1

?
Logged
concordia cum veritate

p.r.

  • Jr. Member
  • **
  • Karma: 0
  • Offline Offline
  • Posts: 50
Re: High CPU usage during procedure of final file hashing
« Reply #43 on: March 23, 2009, 09:08:04 PM »

There is no predefined __powerpc__ at all, neither in native nor cross gcc.
Code: [Select]
#define __powerpc__ 1

?
Or you can try
Code: [Select]
echo x | cpp -dM - | sort to see all predefined symbols.
"echo x | cpp -dM - | sort" don't  show any predefined symbols that could be valid for powerpc-linux-uclibc-gcc (powerpc 603e, linux), only for native gcc (x86, linux)
Logged
Pages: 1 2 [3]