aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Pages: 1 [2]

Author Topic: Charset Support  (Read 11870 times)

phoenix

  • Evil respawning bird from aMule Dev Team
  • Developer
  • Hero Member
  • *****
  • Karma: 44
  • Offline Offline
  • Posts: 2503
  • The last shadow you'll ever see
Re: Charset Support
« Reply #15 on: March 27, 2007, 12:05:54 PM »

pochu,

Today's cvs tarball has the last patch. Please report if your issue is gone.

Cheers!
Logged

pochu

  • Approved Newbie
  • *
  • Karma: 4
  • Offline Offline
  • Posts: 34
Re: Charset Support
« Reply #16 on: March 28, 2007, 06:44:07 PM »

Hi phoenix

I've built today's tarball, and have downloaded three files with special characters. It hasn't worked.
I attach a screenshot of ~/.aMule/Incoming/ . One image is better than a thousand words ;)

I have nothing more to say atm, will try to investigate this further.

Regards
Pochu
Logged

phoenix

  • Evil respawning bird from aMule Dev Team
  • Developer
  • Hero Member
  • *****
  • Karma: 44
  • Offline Offline
  • Posts: 2503
  • The last shadow you'll ever see
Re: Charset Support
« Reply #17 on: April 21, 2007, 09:30:17 PM »

Hi Pochu,

This image you submitted is from a system file browser or something like this, right? This is not from aMule.

The situation is the following: if the file name you have in your directory uses a character set that is not the same as in your system. Your system uses UTF-8. It will always read garbage when you have, for instance, ISO-8859-1file names, which seems to be the case.

Well, we have to find a reasonable policy here. At the present time, aMule works like this:

1) Converting from multibyte to UNICODE:
- Assume that input name is ISO-8859-1 and try to convert it to UNICODE. If this fails, then try to convert from UTF-8 to UNICODE.

2) Converting from UNICODE to multibyte:
- Try to convert UNICODE input to ISO-8859-1. If this fails, convert it to UTF-8.

I have been thinking, and maybe step 1 is wrong. ISO-8859-1 to UNICODE must never fail, while UTF-8 to UNICODE can fail. Maybe the right order is the opposite. Still there could be a situation where the file name is ISO-8859-1 but by chance it was a valid UTF-8 sequence.

What we can experiment with is not use ISO-8859-1file names and using the system encoding instead.
Logged

phoenix

  • Evil respawning bird from aMule Dev Team
  • Developer
  • Hero Member
  • *****
  • Karma: 44
  • Offline Offline
  • Posts: 2503
  • The last shadow you'll ever see
Re: Charset Support
« Reply #18 on: April 23, 2007, 04:07:10 PM »

Tomorrow CVS code will have a different behaviour for #1 above. Now we first try UTF-8 when converting to UNICODE. If this fails, then we try ISO-8859-1.

Cheers!
Logged

pochu

  • Approved Newbie
  • *
  • Karma: 4
  • Offline Offline
  • Posts: 34
Re: Charset Support
« Reply #19 on: April 23, 2007, 04:19:13 PM »

Hi phoenix

Firstly, sorry for not replying to the previous message, I forgot to do it!

Yes, that screenshot is the gnome file manager, nautilus. As you can see, that's the amule Incoming folder, and those are some files I downloaded to test your changes. But it didn't worked.

I'll try tomorrow's tarball, let's see if we can finalize this! (/me testing, you coding ;) )

Best regards
Pochu
Logged

phoenix

  • Evil respawning bird from aMule Dev Team
  • Developer
  • Hero Member
  • *****
  • Karma: 44
  • Offline Offline
  • Posts: 2503
  • The last shadow you'll ever see
Re: Charset Support
« Reply #20 on: April 23, 2007, 04:25:13 PM »

Hi Pochu,

It is not a matter of working or not working. Maybe you did not understand. It cannot work always, that is the problem.

We could add an option to use the system encoding instead of ISO-8859-1... I don't know, I would like to hear other people's oppinion on that.

Cheers!
Logged

pochu

  • Approved Newbie
  • *
  • Karma: 4
  • Offline Offline
  • Posts: 34
Re: Charset Support
« Reply #21 on: April 25, 2007, 02:08:01 PM »

Hi phoenix:

With today's tarball, it still fails with the "ñ" in the files. In the Incoming folder, that char is displayed as "?".

Maybe that option of using the system encoding will be great :)

I've also seen that the next wx release will have UTF-8 support (http://www.wxwidgets.org/wiki/index.php/Development/UTF8)

Maybe with that this will be easier?
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Charset Support
« Reply #22 on: April 25, 2007, 02:33:46 PM »

What, you mean along with the fact that they broke compilation?
Logged

pochu

  • Approved Newbie
  • *
  • Karma: 4
  • Offline Offline
  • Posts: 34
Re: Charset Support
« Reply #23 on: April 25, 2007, 02:59:25 PM »

Kry: the compilation fails with wx2.9, but not with 2.8.x (I've built now against 2.8.4-rc1).

This also failed with previous amule cvs and wx2.8.x (2.8.3.0, e.g.)
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Charset Support
« Reply #24 on: April 25, 2007, 03:53:24 PM »

UTF8 is afaik only in 2.9
Logged

pochu

  • Approved Newbie
  • *
  • Karma: 4
  • Offline Offline
  • Posts: 34
Re: Charset Support
« Reply #25 on: April 25, 2007, 03:57:22 PM »

In wx yes, this can be solved in amule
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Charset Support
« Reply #26 on: April 25, 2007, 04:15:41 PM »

?
Logged

pochu

  • Approved Newbie
  • *
  • Karma: 4
  • Offline Offline
  • Posts: 34
Re: Charset Support
« Reply #27 on: April 25, 2007, 04:22:36 PM »

I mean the charset issue :)

Phoenix got it solved some time ago (with the first or the second patch) but it shouldn't be the first way to do it, and now it doesn't work.

Sorry if I misunderstood your words!
Logged

phoenix

  • Evil respawning bird from aMule Dev Team
  • Developer
  • Hero Member
  • *****
  • Karma: 44
  • Offline Offline
  • Posts: 2503
  • The last shadow you'll ever see
Re: Charset Support
« Reply #28 on: April 26, 2007, 05:39:30 AM »

Pochu,

I have never got it "solved" as you say. If you want to try a system encoding, please, change this line:
In file src/libs/common/ConvAmule.h, line53:
   ConvAmuleBrokenFileNames aMuleConvBrokenFileNames(wxT("ISO-8859-1"));
Change to:
   ConvAmuleBrokenFileNames aMuleConvBrokenFileNames(wxConvLocal);

That is what Kry said to you in the first place. We might add this to the preferences or even better, start checking the LANG environment variable if this becomes an issue.

Cheers!
Logged

wuischke

  • Developer
  • Hero Member
  • *****
  • Karma: 183
  • Offline Offline
  • Posts: 4292
Re: Charset Support
« Reply #29 on: April 29, 2007, 10:05:25 AM »

Logged
Pages: 1 [2]