Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[stoatwblr]

If what you say is true then there's a bug in WxWidget's handling of files which you need to workaround.

The fact that your programming model is broken is not mitigated by the factor of outside problems. There are any number of reasons a machine may be unexpectedly powered down or rebooted and software which can't cope with that is by defnition BROKEN.

The point remains that the file is momentarily being truncated to 0 bytes and there's a window of vulnerability to the file being left that way - which is not good if amuled is set to startup upon reboot - the first thing it does is to write a bogus new config file and then barfs because autoconnection is off by default.


By the way: If you show the same kind of anger management problems  in your dayjob when bugs are pointed out as you do here then it's a wonder you manage to stay employed. I do note that Amule(edit, I put emule previously, that was a typo)  has had a spectacularly large number of appearances on the Bugtraq list and I have to wonder if your $dayjob coding has as many issues.

[Kry]

If what you say is true then there's a bug in WxWidget's handling of files which you need to workaround.

No, the library has to fix it if it's their error.

The fact that your programming model is broken

Which it isn't, we update the config file every time the values change using the interface from the framework

is not mitigated by the factor of outside problems.

There is nothing to mitigate, as there is no bug in our code.

There are any number of reasons a machine may be unexpectedly powered down or rebooted

No, not really.

and software which can't cope with that is by defnition BROKEN.

Guess the linux kernel is broken then. Or your filesystem. Or libc. Or the framework you are using under wx. Or wx itself. My bet is in #2 or #3.

The point remains that the file is momentarily being truncated to 0 bytes

... for certain filesystems and configurations ...

and there's a window of vulnerability to the file being left that way

Only in case of a catastrophic event.

- which is not good if amuled is set to startup upon reboot - the first thing it does is to write a bogus

Not bogus at all, it's the default configuration. A perfectly valid file.

new config file and then barfs because autoconnection is off by default.

Which is a sensible default.

By the way: If you show the same kind of anger management problems

I'm not angry, therefore I don't need to manage anything.

 in your dayjob when bugs are pointed out as you do here then it's a wonder you manage to stay employed.

In my dayjob, where I'm paid to be nice to the occasional brickheaded moron, I am perfectly hypocritically nice to them. However, in my job I don't deal with customer support, so it's a very rare occurrence.

Of course, you have to keep in mind that this is not an aMule bug. Just because the symptom shows in aMule and can be workarounded in aMule doesn't mean it is an aMule bug.

I do note that Emule has had a spectacularly large number of appearances on the Bugtraq list and I have to wonder if your $dayjob coding has as many issues.

This is not eMule, remember? As for my dayjob coding, it's superb. It has to be for such a sensitive area.

Based on the complete stupidity, brickheadness and absolute lack of coherence of your posts, I have to wonder if your $dayjob is test monkey in a chemical treatment lab where they test the effect of throwing bricks at the heads of retarded monkeys to see the IQ decline.

[Vollstrecker]

There are not much situations the can come up.

1. Power fails before you save your changes -> Nothing happens, your changes are lost.

2. Power fails after you saved your changes and the file is written -> Nothing happens, your changes are saved.

3. Power fails after you saved your changes and file is not written -> Nothing happens, your changes are lost.

4. Power fails after you saved your config while writing the file -> When powering up again, journal recovers your old config and your changes are lost, or journal recovers your changes and your changes are saved.

All those 4 points can happen when you write to a new file and rename it to the desired name. If you get any other situation than these 4, you just use the wrong fs.

All applies to automatically written files aswell.

[Kry]

In a hilarous note, I get this bug with my aMule when my kernel "oops"es, along with firefox and KDE and evolution and pidgin and xchat and other applications getting the same config-reset or corruption problem. I don't blame the applciations because I'm not stupid. I blame XFS for missing the recovery, and the kernel for freezing.

My computer never shuts down or restarts in the cold because it has a built-in UPS: it's a laptop. So kernel freezes are the only problems I get that cause XFS to fuck me over and as rare as they are, I can deal with it. When I think I can't, I'll investigate it and propose a filesystem driver patch or change to other FS with better handling of errors. But for now, I'm happy with XFS's speed.

[stoatwblr]

There are not much situations the can come up.

1. Power fails before you save your changes -> Nothing happens, your changes are lost.

2. Power fails after you saved your changes and the file is written -> Nothing happens, your changes are saved.

3. Power fails after you saved your changes and file is not written -> Nothing happens, your changes are lost.

4. Power fails after you saved your config while writing the file -> When powering up again, journal recovers your old config and your changes are lost, or journal recovers your changes and your changes are saved.

Look one posting down -  Krys is seeing it himself with XFS and yet he thinks it's ok.

The problem is the level of journalling used in the filesystem. Only full journalling can guarantee file integrity and it slows disk activity down by between 25-50%. Journalled filesystems as they are deployed today are intended to protect the filesystem - not individual files. There are specific warnings in Ext3 and XFS documentation that the default (ordered)  mode won't save a file which is being written at the time the time the FS goes down.

My FS is ext4 and I was seeing the truncations journalling=ordered (linux default). So far there have been no truncations with full journalling when the kernel oopes (truncation was happening on ext3/ordered too).

NTFS uses ordered journalling (not adjustable) and there are a huge number of reasons why a windows box can go crunch (There's the old chestnut of "unplugged a USB drive" too).

As for his other comment  which boils down to "Other programs behave like this, so why are you picking on me?":

Back in the days when I ran ORBS, I used to get a lot of whining like that ("XYZ is running an open relay, why are you picking on me?"), to which the canned response was "Thank you for the pointer. These are now being investigated and will be listed if found to have security vulnerabilities"

Needless to say I'll be raising bugs against the ones he's mentioned and I've already notified a few people in appropriate places that there's a general issue which needs addressing in a number of OS packages.

The bottom line is that rewriting any file (let alone a config file) in place is "A very bad idea"  and not making a backup is "just plain nuts". Just because lemmings jump off a cliff doesn't mean you should too.

[Kry]

You just can't read. I don't think is ok for XFS to do this to my files. The point is that I know who to blame for it: XFS. That's the reason they warn you about it, because it's their decision not to change it. If you care about file contents this much and have often hard locks or power offs of the system, don't use a filesystem that will destroy your data in those cases. It's that simple. So far I can deal with it (also because I keep my own script backing up sensitive files from time to time, thank you very much), if I decide it's not worth it I'll change it.

But I'll never be as retarded as to ask people to workaround it on userspace.

Now, you mention that "rewriting any file (let alone a config file) in place is "A very bad idea"  and not making a backup is "just plain nuts"". We happen to agree with you! That's why we do backups of every file with a safe function before writing new data to them, when they are sensitive. Of course, given that we use wxFileConfig for abstraction of the file config handling, it's not aMule's business to change the flush() behaviour. LEARN TO READ ALREADY.



Also, again vulnerabilities. You keep using that word. I don't think it means what you think it means.

[Kry]

Incidentally I look a lot like Inigo Montoya

[Vollstrecker]

Look one posting down -  Krys is seeing it himself with XFS and yet he thinks it's ok.

Why don't you just do what you tell me. Read this:

I don't blame the applciations because I'm not stupid. I blame XFS for missing the recovery

You see: You're wrong.

The problem is the level of journalling used in the filesystem. Only full journalling can guarantee file integrity and it slows disk activity down by between 25-50%. Journalled filesystems as they are deployed today are intended to protect the filesystem - not individual files.

So you think it's intended that you have an fs with 100% file integrity and 0 useable files? File integrity means, that the file and the filesystem is in an defined state. Written or not written. Somewhat in between is not defined.

There are specific warnings in Ext3 and XFS documentation that the default (ordered)  mode won't save a file which is being written at the time the time the FS goes down.

I can't see any warning in Documentation/filesystems/ext3.txt, but I see:

/ The Journaling Block Device layer (JBD) isn't ext3 specific.  It was designed
/ to add journaling capabilities to a block device.  The ext3 filesystem code
/ will inform the JBD of modifications it is performing (called a transaction).
/ The journal supports the transactions start and stop, and in case of a crash,
/ the journal can replay the transactions to quickly put the partition back into
/ a consistent state.

and

/ * ordered mode
/ In data=ordered mode, ext3 only officially journals metadata, but it logically
/ groups metadata and data blocks into a single unit called a transaction.  When
/ it's time to write the new metadata out to disk, the associated data blocks
/ are written first.

Which means, that a  journaling filesystem should do exactly what I stated.

My FS is ext4 and I was seeing the truncations journalling=ordered (linux default). So far there have been no truncations with full journalling when the kernel oopes (truncation was happening on ext3/ordered too).

There has been no truncations, but they happend on other fs, too?

NTFS uses ordered journalling (not adjustable) and there are a huge number of reasons why a windows box can go crunch

You can fuck up a windows box in uncounteable ways. What's your point?

(There's the old chestnut of "unplugged a USB drive" too).

Beside the fact, that USB write actions are buffered and you can plug the drive out before any action is started, it is still a problem of the fs. This time multiplied with the fact that the user can influence the participating parts and missbehaves, because he doesn't eject the drive the right way.

As for his other comment  which boils down to "Other programs behave like this, so why are you picking on me?":

Back in the days when I ran ORBS, I used to get a lot of whining like that ("XYZ is running an open relay, why are you picking on me?"), to which the canned response was "Thank you for the pointer. These are now being investigated and will be listed if found to have security vulnerabilities"

Which in this case is no problem of the software, the admin just didn't it configure right. What the hell has this to do with our it's the problem of another software that is used discussion?

Needless to say I'll be raising bugs against the ones he's mentioned and I've already notified a few people in appropriate places that there's a general issue which needs addressing in a number of OS packages.

Needless to say that the maintainers will forward the bugs to upstream, because it's a generall problem of the software and not of the packaging.

The bottom line is that rewriting any file (let alone a config file) in place is "A very bad idea"  and not making a backup is "just plain nuts".

So you're bottom line is, that we should rewrite major parts of a toolkit we use, which we use just because it provides the functionality we need and we don't want to rewrite this all, plus we should reimplement the fs-drivers which fuck up their job. You should know that first there are not just 2 fs's out there, second not all are free and third this will multply the size of the binaries, and you will be the first that is whining about the huge memory usage and your "expert programmers" will tell you to tell us, that there are shared libs and framworks that can be used todeal with such stuff.

Just because lemmings jump off a cliff doesn't mean you should too.

When you direct the first lemming over a bridge, the others will follow, too.

Go dying.

[Kry]

I do note that Amule(edit, I put emule previously, that was a typo)  has had a spectacularly large number of appearances on the Bugtraq list and I have to wonder if your $dayjob coding has as many issues.

I just saw your ninjaedit. You're right, aMule has a spectacularly large number of appearances on bugtraq: four.

One of them is a security problem with wxExecute that could cause a problem if the user previews a file that has text like "'; rm -rf /" at the end (an extreme example, it can be any shell command).  It was fixed in 2.2.5.

Another one is related to the fact webserver images could be guessed without logging in three years ago, published after we already released a fix.

A third one isn't in aMule but in aStats, and it's from 5 years ago!

And the last one is some dude asking how to find viruses in p2p networks and someone mentioning using aMule to search for them.

So yeah, a SPECTACULAR amount of appearances. I'm humbled by your words, good sir!

[wuischke]

My FS is ext4

No offence, but here's your problem. I've had plenty of weird data losses (fortunately nothing serious) with ext4, too and we're not the only ones, if you search the internet.

It's a file system still in development and just not ready yet. (I'm speaking of my own experience with 2.6.29 ext4, I know that for instance the Fedora devs think differently.)

[atilla]

Main problem is EXT4 FS "delayed allocation" bug.  I hope it is fixed in 2.6.30 kernels. An automatic configuration backup feature can be requested for Windows systems, if devs agreed then it will be added to aMule. But in Linux there is a easy way:

Code: [Select]

cp $HOME/.aMule/amule.conf $HOME/.aMule/amule.conf.backup && amule


P.S.: Kry, please calm down man. Any of aMule users won't want you to get a heart attack. 

[Kry]

That's a common misconception. I am pretty calm when writing those posts

[Kry]

And so the thread gets wrapped with a single comic strip about how aMule is coded.

[stoatwblr]

This explains the filesystem issue, and WHY it's primarily an application problem.

https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/45

This isn't just aMule, a lot of programs wrongly assume that what's written to the disk is automatically committed immediately and just because other people make bad assumptions and write code which is vulnerable to config file trashing doesn't follow that you have to follow suit - trying to have the FS second guess what needs immediate committal will probably result in the same kind of performance issues that happened when Firefox 3.0.0 used fsync() for every snigle write

The problem is particularly noticeable with aMule because it writes to disk so much.

Short answer is use fdatawrite() to commit to disk, or fsync() if that's not available

Long answer is write newconfig; mv config config~; mv newconfig config; fdatawrite().

Just because Wxwidgets gives you a tool to blow your leg off (file rewriting in place) doesn't mean you should use it, and the fact that the file is showing up truncated shows that while you may think it's editing the file in place, it's clearly not doing that at FS level.

[wuischke]

May I ask why your computer is crashing so often in the first place? Do you live in a zone with an unstable power supply?

Anyway, we make backups of other important files, so why not backup the configuration file, too? I have to go now, but I'll implement it later. (Unless I forget to...)