MOO2 graphics viewer

[Wolverine]

Hello! I'm new to this forum. LordBrazen told me to go here, so I've registered just to tell you about my recent work. I've managed to crack MOO2 graphics format. It wasn't very hard really. It took me just under two days. I've began writing a simple MOO2 graphics viewer under Python on my Linux machine. I'm not going to publish it until I figure out a few quirks within the format itself, but when I'm done I'm going to release it under GPL license so anyone could make use of it and perhaps even write an editor based upon my work. Until then point your browser to my blog http://www.karoltomala.com/blog/ and check out some chewy screens and technical information regarding the format and the viewer.

Let me know if you're interested and how do you feel about this. Greetings, Wolverine

PS. And here's one of the screens for your amusement:


PS2. I'm also looking for people with knowledge of MOO2 palette formats. PM if you have some info.

[Wolverine]

Hello everyone

I thought some of you may be interested in the progress I have made so far with my MOO2 Graphics Viewer application. I have posted new information on my blog. You can read about it here http://www.karoltomala.com/blog/2007/02 ... s-changes/. There were some problems with my server recently, but I have finally fixed them, so hopefully you won't have problems accessing the site.

I have posted few screens. Any comments regarding the app would be greatly appreciated. And here's one of the screens for you. For more go to my site.



Hope you like it!

[Matthew]

Looks good.

[Wolverine]

Just a quick note to all you crazy MOO2 fans. Take a look at this thing:

And then go to my blog http://www.karoltomala.com/blog/ to download first public alpha of my crappy software. Details are on the site, but for some of you who want to browse the code and perhaps learn something about graphic formats used in MOO2 are encouraged to download it. Unfortunatelly it's not user friendly right now, but I won't have time to code it perhaps for the next few weeks.

As I'm aware it's probably the first attempt at MOO2 graphic moding ever. I hope you like it.

All basic functionality is working in the code, you just have to download and make some use of it. I'm personally waiting for Star Trek mod in the near future.

I have to thank Grig de Griz (the creator of the awesome MOO2 Workshop) for few things that helped me in the development of this app. It's far from being finished and I think it won't be used much right now. However maybe some of you will find it useful.

If you'd like to help me in development I would really appreciate if some of those skilled coders here would send me patches. The code is open, guarded by GPLv2 so you take and you can give if you want to.

Greetings to all of you, let's keep the MOO2 scene alive. This game deserves it

[Zieman]

So you can import graphics to the game too?
WOW!

[Wolverine]

Yes, that's right. You can download the source code from my blog if you want to. I forgot to mention this, but you'll also need QT and PyQT to run the viewer. Functionality classes could be used without QT, if you want to use Converter class you'll need Python-Imaging Library. Converter currently exports 8bit palette PNG files from graphics, and it also can converter 8bit paletted PNG files to MOO2 graphic files, that then could be packed with LBX class from the package to create proper LBX to use with MOO2.

However if you want to do this you'll need to unpack LBX to some directory first, then convert desired PNG's to graphic files and substitute the ones in the directory with your newly created graphics. Be aware that it's good to name those substituted graphics as the original depacked ones, or you'll screw up your archive. It will still build properly, but MOO2 will use rearanged files in the archive instead of the desired ones. You'll get messed up graphics probably, I haven't tested what will happen in such case.

The most significant problem is that I'm really guessing which palettes are used when and for what graphics, so without MOO2 source code it's a trial and error path. This problem does not occur when the graphic is using internal palette. You can build your own indexed palette with Gimp for example from the desired image, restrict it's color map to desired number (80 colors for PLANETS.LBX for example), a then use savegfx("input.png","output.lbx_0000",palcolor=80) function from Converter module to convert. You can then run main.py which will launch the viewer to see if it has been properly converted. Then you just have to create LBX archive.

Unfortunatelly the viewer needs some internal path editing, because I haven't written any configuration file for now. It's really a pre-alpha version, so you'll need basic knowledge of programming to run it. But it works. If you'll have any questions, feel free to ask. I'll try to answer them.

[Matthew]

I took a quick peek the code. Nice little piece of work. From looking at some other MOO2 code, it looks like there are a lot of custom palettes so one load palette function may not work for all the LBX files. I think the reason for this is because of the graphical limitations they were working with. Since they wanted different color themes for the different screens they made a lot of different color palettes for the artist to draw from. If it were being rewritten today by commercial developers none of that would be necessary.

[Wolverine]

First of all thank you. It's far from being finished, but as I have said it's a pretty good start for editor or perhaps MOO2 remake.

From looking at some other MOO2 code, it looks like there are a lot of custom palettes so one load palette function may not work for all the LBX files.

Yes you're right, but it's not only that different palettes are used for different LBX files. There are internal palettes in some images. There are few palettes mixed with mouse cursors in FONTS.LBX and IFONTS.LBX. But! Take for example COUNCIL.LBX. The first file which in fact is mouse cursor has internal palette. The rest of the graphics in this file doesn't have internal palettes, but neither looks good with so called external palettes (FONTS.LBX and IFONTS.LBX). But if you take the internal palette from the first image in COUNCIL.LBX, other images look good!

It's not the only rather ugly thing that is seen in the graphics. IMO I think that some coders were smoking too much dope. I know that it was the time were size mattered, but this really could be done cleaner. For example. Take SHIPS.LBX. None of the ship images looks good with external palettes. But this is more complicated issue, because there are many small files, one by one pixel size, that doesn't do anything that are really palette holders. BUT! Those are not full palettes, but rather chunks merged with other palettes. The SHIPS.LBX is structured like this: SHIP_GRAPHIC1, SHIP_GRAPHIC2, ... , SHIP_GRAPHICn, PALETTEHOLDER, SHIPGRAPHICx1, SHIPGRAPHICx2, ... , SHIPGRAPHICxn, and so on. Those ships are categorised by colour. (I've only recently spotted that ship design DOESN'T belong to RACE, but to COLOR you choose in the game!) You can think about those ship groups as colour groups. Also if you take a look at some external palettes you can see there are few of them that obviously have the same colour representation for few places in the palette. These are probably the places where those palette chunks from internal palettes go (are merged). This is not a stupid idea, but to check which palettes are used for ships for example you'll have to get numerical representations of the colours in the ship graphics and then compare with those placeholder places in external palettes.

That's why a full blown converter and editor would need external representation of palette relations between most of the graphic files, external palettes and internal (merged) palettes in some kind of XML file. I want to do it, but currently my business projects are priority, so I won't have time for it. If anybody would like to decipher which palettes are used for what and where it would be really nice and could really speedup the development of the converter and the viewer.

[Matthew]

First of all thank you. It's far from being finished, but as I have said it's a pretty good start for editor or perhaps MOO2 remake.

From looking at some other MOO2 code, it looks like there are a lot of custom palettes so one load palette function may not work for all the LBX files.

Yes you're right, but it's not only that different palettes are used for different LBX files. There are internal palettes in some images. There are few palettes mixed with mouse cursors in FONTS.LBX and IFONTS.LBX. But! Take for example COUNCIL.LBX. The first file which in fact is mouse cursor has internal palette. The rest of the graphics in this file doesn't have internal palettes, but neither looks good with so called external palettes (FONTS.LBX and IFONTS.LBX). But if you take the internal palette from the first image in COUNCIL.LBX, other images look good!

It's not the only rather ugly thing that is seen in the graphics. IMO I think that some coders were smoking too much dope. I know that it was the time were size mattered, but this really could be done cleaner. For example. Take SHIPS.LBX. None of the ship images looks good with external palettes. But this is more complicated issue, because there are many small files, one by one pixel size, that doesn't do anything that are really palette holders. BUT! Those are not full palettes, but rather chunks merged with other palettes. The SHIPS.LBX is structured like this: SHIP_GRAPHIC1, SHIP_GRAPHIC2, ... , SHIP_GRAPHICn, PALETTEHOLDER, SHIPGRAPHICx1, SHIPGRAPHICx2, ... , SHIPGRAPHICxn, and so on. Those ships are categorised by colour. (I've only recently spotted that ship design DOESN'T belong to RACE, but to COLOR you choose in the game!) You can think about those ship groups as colour groups. Also if you take a look at some external palettes you can see there are few of them that obviously have the same colour representation for few places in the palette. These are probably the places where those palette chunks from internal palettes go (are merged). This is not a stupid idea, but to check which palettes are used for ships for example you'll have to get numerical representations of the colours in the ship graphics and then compare with those placeholder places in external palettes.

This type of thing used to be common when memory and disk space was in short supply. The developers were trying to squeeze as much as possible out of the resources available at the time. Probably what they did was to make an LBX packer tool that compressed everything as tightly as possible.

From looking at the MOO2 disassembly, it looks like then they wrote individual functions for accessing each type of LBX file.

I'm learning assembly language so that I can better understand the disassembly. It's slow going. Assembly is totally different from learning something like C++ or Python.

That's why a full blown converter and editor would need external representation of palette relations between most of the graphic files, external palettes and internal (merged) palettes in some kind of XML file. I want to do it, but currently my business projects are priority, so I won't have time for it. If anybody would like to decipher which palettes are used for what and where it would be really nice and could really speedup the development of the converter and the viewer.

That's a pretty tall order. Someone else might help, who knows.

There's no rush since it's purely a volunteer project. It doesn't matter if it gets done in a month or several years from now.

One thing that I learned in college is that real-life has to come before gaming or else you'll stay stuck at the bottom never truly succeeding at anything besides games. Being godly at gaming is pretty meaningless unless you decide to become a pro-gamer or game developer.

[Wolverine]

This type of thing used to be common when memory and disk space was in short supply. The developers were trying to squeeze as much as possible out of the resources available at the time. Probably what they did was to make an LBX packer tool that compressed everything as tightly as possible.

Not really. It's not compressed as tightly as possible. LBX archives are simple concatenated files with additional header with offsets pointing each file. It's much more like tar (but simpler) than real compressed archives like gz or zip.

From looking at the MOO2 disassembly, it looks like then they wrote individual functions for accessing each type of LBX file.

Every LBX (except movies) are exactly the same structure. Individual files within the archive are what is different and if you've taken a look at disassembled code you should know that LBX decompression is the same for each archive, only the routines accessing individual files are different. That's what I think it looks like, because I don't have disassembled code. And BTW. I wasn't interested in other types of files other than palettes and graphics so I won't discuss them here.

I'm learning assembly language so that I can better understand the disassembly. It's slow going. Assembly is totally different from learning something like C++ or Python.

As for assembly language. It's funny and really cool. There are many resources on the net discussing this language. Yes it's different, because it's the machine code, but C/C++ compiles it's code to assembly before linking too. Take a look at Netwide Assembler. In my opinion learning assembly with NASM and Linux is much more fun and it's better documented than DOS equivalent. And they really are not very different, except for registers and interrupt request numbers. There's pretty good assembly site at asm.sf.net.

That's a pretty tall order. Someone else might help, who knows.

There's no rush since it's purely a volunteer project. It doesn't matter if it gets done in a month or several years from now.

One thing that I learned in college is that real-life has to come before gaming or else you'll stay stuck at the bottom never truly succeeding at anything besides games. Being godly at gaming is pretty meaningless unless you decide to become a pro-gamer or game developer.

Well. For me it matters. It's my ambition to finish it. And as for real life. I won't agree with you. It's not only about the gaming, but most importantly it's great fun to crack the format and then write a program utilising it. But there's also another side to it. I'm gaining more and more knowledge in programming, cracking and methods of data representation and deciphering. I'm a self-taught network administrator and programmer, so this significantly adds to my skills. And if I ever want to be really good at what I'm doing I have to gain vast knowledge about data transmission, representation, cryptoanalysis, signal interception and decoding. And most of all this is what I do in my real life too. So writing this little app might look like it's not related to my work, but trust me. It is.

[Matthew]

This type of thing used to be common when memory and disk space was in short supply. The developers were trying to squeeze as much as possible out of the resources available at the time. Probably what they did was to make an LBX packer tool that compressed everything as tightly as possible.

Not really. It's not compressed as tightly as possible. LBX archives are simple concatenated files with additional header with offsets pointing each file. It's much more like tar (but simpler) than real compressed archives like gz or zip.

I know it's not gziped or anything like that but it is tightly packed.

From looking at the MOO2 disassembly, it looks like then they wrote individual functions for accessing each type of LBX file.

Every LBX (except movies) are exactly the same structure. Individual files within the archive are what is different and if you've taken a look at disassembled code you should know that LBX decompression is the same for each archive, only the routines accessing individual files are different. That's what I think it looks like, because I don't have disassembled code. And BTW. I wasn't interested in other types of files other than palettes and graphics so I won't discuss them here.

I haven't looked too deeply at the LBX format yet but this sounds correct.

I'm learning assembly language so that I can better understand the disassembly. It's slow going. Assembly is totally different from learning something like C++ or Python.

As for assembly language. It's funny and really cool. There are many resources on the net discussing this language. Yes it's different, because it's the machine code, but C/C++ compiles it's code to assembly before linking too. Take a look at Netwide Assembler. In my opinion learning assembly with NASM and Linux is much more fun and it's better documented than DOS equivalent. And they really are not very different, except for registers and interrupt request numbers. There's pretty good assembly site at asm.sf.net.

It is interesting to learn assembly but it is somewhat time consuming. I'm more of a high-level OOP programmer than a machine level coder.

I'm also learning Eiffel using the new IDE that is lincensed under the GPL.

That's a pretty tall order. Someone else might help, who knows.

There's no rush since it's purely a volunteer project. It doesn't matter if it gets done in a month or several years from now.

One thing that I learned in college is that real-life has to come before gaming or else you'll stay stuck at the bottom never truly succeeding at anything besides games. Being godly at gaming is pretty meaningless unless you decide to become a pro-gamer or game developer.

Well. For me it matters. It's my ambition to finish it. And as for real life. I won't agree with you. It's not only about the gaming, but most importantly it's great fun to crack the format and then write a program utilising it. But there's also another side to it. I'm gaining more and more knowledge in programming, cracking and methods of data representation and deciphering. I'm a self-taught network administrator and programmer, so this significantly adds to my skills. And if I ever want to be really good at what I'm doing I have to gain vast knowledge about data transmission, representation, cryptoanalysis, signal interception and decoding. And most of all this is what I do in my real life too. So writing this little app might look like it's not related to my work, but trust me. It is.

I have no doubt that it could be useful depending on what career path you take. When I first graduated from college the recruiters mostly just seemed interested in my degree and grade point average. They also tested my programming ability and understanding of computers. Getting good references from the professors seemed to help too.

If you become super proficient at the low-level technical stuff you should keep it to yourself until it's needed. The term 'hacker' scares many of the managers (or customers) away because they're worried about security.

[Wolverine]

I have no doubt that it could be useful depending on what career path you take. When I first graduated from college the recruiters mostly just seemed interested in my degree and grade point average. They also tested my programming ability and understanding of computers. Getting good references from the professors seemed to help too.

If you become super proficient at the low-level technical stuff you should keep it to yourself until it's needed. The term 'hacker' scares many of the managers (or customers) away because they're worried about security.

This discussion is a getting a little bit off-topic. I'm not into 'hacking' or something. I just want to be a good at what I'm doing. Writing programs like MOO2 Graphics Viewer is most of all just pure fun. If it turns out to be useful in future I'd be even more happy.

As for managers that are scared because of the word 'hacker'. You might be right that there's little understanding of IT as whole, but here in Poland if you're working for international company they're embracing people with great skill and knowledge. I have once worked for such company and one person from the board of managers asked me if I could try to somehow access the old accounting program that they forgot password to. I agreed, got on the computer and managed to access this program within few minutes with CEO standing right behind me. When I did it few people who were looking at what I'm doing said: "WOW". Besides that they were ignorant in this topic (it wasn't very hard to do) this, and few other things I did there gave me quite powerful position in that company. What's the point of this story? It's all about making impression, so if you're into developing your career quite fast, just show your knowledge and skills to the right persons at the right time. Anybody can do that. I hope you'll manage do this easily too.

And besides. MOO2 is my favourite game of all times, despite the fact that I got onto it just four years ago. There's only three games that are worth playing for me: MOO2, OpenTTD and System Shock 2. I'd really love to see remake of MOO2 which has more technologies, better and smarter AI fully utilizing diplomacy, intrigues, spies, maybe better graphics, more advanced diplomacy system and better strategic options like for example creating traps, key planetary systems that would guard the access to the rest of empire with better fortifications, starlanes would certainly add a strategic value, less micromanagement in the late game and much more. Maybe the work done here by Grig de Griz and me will spark a movement for creating such remake. I have high hopes for it, because MOO2 is one of the few games that simply cannot be substituted.

[Matthew]

I have no doubt that it could be useful depending on what career path you take. When I first graduated from college the recruiters mostly just seemed interested in my degree and grade point average. They also tested my programming ability and understanding of computers. Getting good references from the professors seemed to help too.

If you become super proficient at the low-level technical stuff you should keep it to yourself until it's needed. The term 'hacker' scares many of the managers (or customers) away because they're worried about security.

This discussion is a getting a little bit off-topic. I'm not into 'hacking' or something. I just want to be a good at what I'm doing. Writing programs like MOO2 Graphics Viewer is most of all just pure fun. If it turns out to be useful in future I'd be even more happy.

As for managers that are scared because of the word 'hacker'. You might be right that there's little understanding of IT as whole, but here in Poland if you're working for international company they're embracing people with great skill and knowledge. I have once worked for such company and one person from the board of managers asked me if I could try to somehow access the old accounting program that they forgot password to. I agreed, got on the computer and managed to access this program within few minutes with CEO standing right behind me. When I did it few people who were looking at what I'm doing said: "WOW". Besides that they were ignorant in this topic (it wasn't very hard to do) this, and few other things I did there gave me quite powerful position in that company. What's the point of this story? It's all about making impression, so if you're into developing your career quite fast, just show your knowledge and skills to the right persons at the right time. Anybody can do that. I hope you'll manage do this easily too.

And besides. MOO2 is my favourite game of all times, despite the fact that I got onto it just four years ago. There's only three games that are worth playing for me: MOO2, OpenTTD and System Shock 2. I'd really love to see remake of MOO2 which has more technologies, better and smarter AI fully utilizing diplomacy, intrigues, spies, maybe better graphics, more advanced diplomacy system and better strategic options like for example creating traps, key planetary systems that would guard the access to the rest of empire with better fortifications, starlanes would certainly add a strategic value, less micromanagement in the late game and much more. Maybe the work done here by Grig de Griz and me will spark a movement for creating such remake. I have high hopes for it, because MOO2 is one of the few games that simply cannot be substituted.

Didn't mean to sound preachy or to go too far off of topic but learning how to crack a program is in the realm of hacking. There are good hackers and bad hackers. I wasn't suggesting that you are the bad kind. There isn't anything wrong with trying to improve this game by making modifications to it. I think the graphics viewer/changer is a good idea.

I'm not sure how companies work in Poland but here in the USA many of the larger corporations are worried about protecting their intellectual property. They go to great lengths to insure that security is tight. To get a job you have to go through a thorough background check and answer a lot of questions. If they think that you could circumvent their security systems they would be less likely to let you in unless they knew they could trust you. Areas where they like people that know how to hack is in network security and in reverse engineering projects. The catch is that you need a high level of security clearance to get one of those jobs.

[Sogroon]

Hi! Is there any breakthrough in this project?

[Time]

Yes, back on topic.
How's it going?
Have other mods been created?
A Star Trek, as someone mentioned?
A Stargate SG1 version maybe?
Babylon 5?
Others?

Have you worked out the bugs you mentioned?

Is there a user friendly version available?

Love that last one, reminds me of Dune.

Keep us posted plz.