Hi, ppclover.
In order for a program on the Mac to be a full-fledged application, with proper GUI integration, it must be wrapped in an application package. The behavior you're seeing is typical of trying to run an un-packaged GUI program from the terminal.
This packaging is done for the monolithic aMule GUI with the aMule.app package, but we haven't provided such a package for amulegui. You can adapt the aMule.app package for amulegui. Copy aMule.app to aMuleGUI.app (or whatever you would like to name it). Copy the amulegui program into the Contents/MacOS subdirectory. You should probably delete all of the other items from Contents/MacOS, mostly just to save space. Now, edit all of the other files inside the new aMuleGUI.app and change references to "amule" into "amulegui". Most important is the CFBundleExecutable key in the Info.plist. The value of this key names the item in Contents/MacOS which gets run when you run the application.
OK, to save you the trouble, I have created a suitable aMuleGUI.app package and attached it to this post. Note: there
is no amulegui executable in this package, you have to put yours into it.
By the way, as near as I know you are the first person to try running amulegui on the Mac.

Let us know how it works.