Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[ZombieNinja]

Hi,

Here is a scenario that I'd propose:

One client (let's name this client 'a') is downloading file "xyz.data",  a remote aMule client "b" connects to client 'a', but instead of downlooading data from 'a', client 'b' wanted to query the content of "xyz.data".

I want to have a small database that sits on top of aMule such that, when the database is aware of the partial file downloaded by the aMule (while amule is running of course) and can allow remote database clients (that also sit on top of amule) to query the partially completed file.

I'm quite new to the source code of aMule, so I would be much appreciated if anyone can point out to me which part of aMule source I should modifiy to export teh following information to other libraries:

1) the amount of data that's been downloaded.
2) telling remote client which part of the orginal file does the current client have.
3) send messages between clients, such that my database can listen to it.


Any help is much appreciated.

regards.

[lfroen]

query the content of "xyz.data".

What exactly is "content" ? How does it different from downloading xyz.data ?

2) telling remote client which part of the orginal file does the current client have.

That's what ed2k protocol is about - clients exchange partfile information (what part of file each one have).

1) the amount of data that's been downloaded.

This is actually same as (2)

Please explain what are you talking about.

[ZombieNinja]

What I try to do is to have a database system, which handles the contents of the shared data.

For example, if you have some DNA data files that the database system recoognizes and say if the database system has the capability to allow users to query or extract informatio from these data data, rather than just downloading data.

Based on that, I want to harness ed2k and kad to do the transfer of data and queries to other peers, such that when a client issues a db query on finding me the DNA data about Gene 123, any peers that have this information can answer my query and return the details of DNA data to me, rather than telling me that the file (which could be gigabytes or terabytes) is at this location.

The idea is some other peers (note that other peers may also have the database system sitting on top of aMule) may be doing the following things:
1) downloading the DNA file
2) querying the orlginal source about other DNA data.

but if the current client also issues a query which is regarding to the same file, but different content. The chances are that these peers may contain partial results for my query and I can speed up the query process by retreiving data from multiple sources.

[lfroen]

Well, this is not exactly what file sharing does. As the name suggest, file sharing shares files which mean that protocol operates on files and their hashes and offsets. Goal of protocol is to correctly transfer files. What you need is distributed database, such that runs simultaneously several queries and joins results.
While having similar implementaion details (peers discovery, handshaking protocol, auth etc) those two have different different concepts and as a result, they are different applications.
Hope I answered your question.

[ZombieNinja]

I know that AMule is a file sharing application using the file sharing protocol. I also understand the concept of distributed database, this is one of my research area.

My experiments is to see if I can harness the P2P network to carry out distributed query on partial files rather than relying on the architecture of a distributed database sytsems (i.e., without update synchronization, handshaking, etc <--- this is just too much overhead).

What I wanted to know is where in the sources code should I look at to modify a few things so I can make aMule allow other clients to query the file contents.


the rest I will take care of.

cheers.

[phoenix]

ZombieNinja,

As far as I understand, you will have to extend the protocol to do that. Or even better, modify aMule or develop a pluggin, so that only your mod is database query aware.

I kinda agree with lfroen, what you need is a distributed database. What is the gain of having p2p + distributed database intermixed? If you are searching for a specific part of the database, it is more efficient to transfer only that part. Like lfroen said, ed2k protocol goal is to correctly transfer files.

It is a little difficult to point you the exact places in the source code. You will probably want to look at the SharedFilesList and DownloadQueue. The code is big, and things are not quite well distributed...

Anyway, good luck! Cheers!

[ZombieNinja]

Hi,

Thanks for your advice, that's what I was intended to do, making a mod of aMule.

I was being told to post it in this forum to get some advice. So it's not exactly a FEATURE requrest for aMule itself.

I think you guys might have misunderstood me on that one.

But yes, I just wanted to check how it would perform without the synchronisation servers that distributed databases use.

thanks.

I will let you know how my experiments go on this.