Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[theWinterLion]

Good day to everyone.  I would like to use aMule as the heart of a music recommendation service because aMule is multi-platform, and has a CLI version.  You can look at what is planned by checking out iRate Radio and "Share the Music" on SourceForge.

I have a few questions as to aMule's suitability, and would like to solicit the aMule community's wisdom.

1. aMule in Share the Music will distribute (potentially) millions of small text files (10-100 kb).  Can aMule handle that many unique files?

2. What is the status of aMule's UPnP?  I see it as "in development" some places, but those reports are quite old.

3. If a client was firewalled, could they still share millions of small files effectively?  In other words, because the files are so small, peers would only need to find one other peer sharing that file to receive it completely.  For example, if there were a million users and 60% did not have UPnP configurable routers, would the network still work effectively sharing a million 10kb files?

Thanks in advance for any information you can provide.

Kind regards,
Rob Peters

[lfroen]

1. aMule in Share the Music will distribute (potentially) millions of small text files (10-100 kb).  Can aMule handle that many unique files?

Millions? Each file is identified by 16-byte hash. So, 1M files will take 16M hashes. Looks like bad idea to start with.

2. What is the status of aMule's UPnP?

Working.

3. If a client was firewalled, could they still share millions of small files effectively?

You can't share millions of files (no matter how small) effectively.

[theWinterLion]

Thank you for your reply, lfroen.  May I ask for some further clarification?

1. aMule in Share the Music will distribute (potentially) millions of small text files (10-100 kb).  Can aMule handle that many unique files?

Millions? Each file is identified by 16-byte hash. So, 1M files will take 16M hashes. Looks like bad idea to start with.

2. What is the status of aMule's UPnP?

Working.

3. If a client was firewalled, could they still share millions of small files effectively?

You can't share millions of files (no matter how small) effectively.

As to question 1, I'm not sure what the issue is you're raising.  There are 2^64 MD4 hashes, no?  Surely 1,000,000 hashes won't be an issue.  And why would there need to be 16 hashes for every file?

As to question 2, that's good news.  Thank you.

As to question 3, I've done some further reading on what having a lowID means.  Thanks.  As to sharing millions of files effectively, you mean that one person could not share millions of files effectively, yes?  Okay, that's fine.  I just fired up eMule, and there are definitely millions of files on the network, so that will suit my purposes.  I had been wondering if any one client could have a million files, but it was more a question of curiosity (as in, "How many could a single peer share?")

Thanks again for your reply.

[Kry]

The max files a user can share effectively is in the range of hundreds. You can't push millions of files in the ed2k/kad network before either the servers or kad developers start effectively blocking your traffic for abusing the network.

You should develop a independant version of the kad network for your clients, with a different protocol number and some central server with the bootstrap nodes. Keep in mind that a kademlia-based network is going to need several hundreds, if not thousands, of users to work effectively ad have a proper redundancy against node shutdown.

[theWinterLion]

Thank you, Kry.

Is it possible to change aMule such that it will only communicate with peers in my network (i.e. not with the main eDonkey network)?  I certainly do not want to harm the existing Kad and eDonkey networks.  It would be perfectly acceptable (even preferable) if the peers running my software communicated only with other peers running my software.

If I have to develop my own p2p program, I will, but I'd much rather use something that already exists.

Kind regards,
Rob

[Kry]

It depends on whether you want to have your own ed2k server for your network, if you want to use a Kad-based network, etc.

Could you actually handle the load of a ed2k server that indexes millions of files?

[theWinterLion]

Thanks again for your reply, Kry.

I would prefer a Kad network for several reasons.

1. There is no searching, so response time for finding peers is not critical.

2. As you mention, it will allow me to (if I understand it correctly) not have to rely on an ed2k server.

3. It is more robust than any solution I could code.  I could make something to distribute the tiny files, but I'd like to be able to d/l full albums and lossless songs and so forth.  I don't want to reinvent the wheel.

Is it possible to use aMule in a Kad-only network that would be exclusive to other Kad networks?

Kind regards,
Rob

[Kry]

Not OOB, but modifying it for it would be quite easy.

[theWinterLion]

Thanks for your reply, Kry.

That is good news.  I think the best option is the one you outlined below (in your post, "You should develop an independent version...").

Could you explain what you mean by, "with a different protocol number"?  Is there a protocol number hard-coded into aMule?  By changing this number, could I create a seperate aMule network?

Kind regards,
Rob