Thanks to the developers! (& suggestion for savegames)

[nick]

Hi,

I played the original Elite, Frontier and Frontier 2 when they were released - their atmosphere and immersion is legendary. I recently discovered Pioneer, and it's the game I've long awaited :)

Real distances, newtonian physics, an enormous procedurally generated universe and modern PC power gives a captivating experience. A feature of Pioneer I am particularly fond of is the ability to do proper exploration of planets and moons in uncharted systems. A big thank you to the developers for the time dedicated to bringing this game to its current impressive state. I may be able to help with the project at some point (I have C++, physics and computer modelling experience, but no games programming experience).

Compatibility of savegame files with future releases is a feature that will greatly enhance Pioneer - in the sense that players will know that the time spent in-game will not be lost (my savegame became incompatible with latest release in early August, so I've got to start again, or forfeit all new features released since then). I did look at the source code to see if I could figure out where the save/load game code was. It's probably much more tricky than I'm thinking - when loading an older savegame file, when attempting to read some data that doesn't exist in the file (added in the newer version) then the code would have to use a default value for the data. So all new data would have to be accompanied by a "default" value.

I reckon that not all data would need to be loaded from an old save game file - player stats, credits, location, ship and equipment would probably do. Everything else could be default newgame data.

To my understanding, there is no header in the binary save game files, and the bytes have to be read in order. Something that is within my ability, is to write a program that reads a savegame file from version X and creates a savegame file for version Y. Is it possible for me to obtain the savegame file format for versions X, Y etc?

I'd need something like:
<data description> | <num bytes> | <type> | <array size> | <default value> (if this is new data, i.e. not saved/loaded in older versions)
note: num bytes = sizeof(type) * array size
e.g.
player name | 64 | char | 64 | no need for default value
credits | 4 | long | 1 | no need for default value
some new data | 32 | long | 8 | default value: 0, 0, 1, 10, 0, 1, 3, 1000
more new data | 8 | double | 1 | default value: 42.81

I think most savegame data could be copied directly from old to new file, but some might need parsing into variables, processing and then written to new file.

Is it easy for me to get hold of this information? My apologies if there's more to it than I'm thinking!
The savegame file conversion program would be for windows, but anybody would be welcome to the source code.
I'd be willing to have a go at writing a backwards-compatible save/load module for Pioneer, but I suspect there's much more to it than I'm aware of, plus I'd need some help in getting started. A savegame conversion program seems a more realistic task for me. If it works, then it may be possible to integrate it with Pioneer itself.

Please let me know if what I suggest is do-able (or not).
Many thanks to whoever has dedicated time and energy to Pioneer :)
Cheers,
Nick

[FluffyFreak]

Hi Nick,

First of all thanks :) it's really good to know that people appreciate the game.

There's been some work done on converting the save system to use JSON, I think that it was put on hold whilst we were looking to merge an especially big and scary feature.

However I don't believe the developer who started it (robn) is doing anything on Pioneer at the moment due to his day job, life, etc so you'd probably be welcome to pick it up.

Most of the developers on Pioneer are just regular folk, some comic and TV artists, a few full time programmers, a mixture of students, the odd retiree :) Join in, it can be fun!

Andy

[nick]

Hi Andy,

Thanks for your reply. Will have a look at the json-savefile branch and see if I can figure it out :)

Nick

[FluffyFreak]

Great! Ask any questions you need either here or on the IRC channel, there's a lot of code, some of it's great, a lot more of it is terrible ;)

So don't feel too overwhelmed by anything, we can either say why something is like it is, or it's probably just evil code and needs rewriting!

Andy

[impaktor]

Welcome!

The save-file bumping is definitely something we need fixing, badly, and I hope you're the one to do it. :-)

And if you feel like you've bitten of more than you can chew with this "save file project", we would all love to see yo join in with any other contribution no matter insignificant it might seem to you.

Karl

[nick]

Thanks for all your replies, and encouragement! :)

I think I'll have a go at the Pioneer json-savefile instead of the savegame-file-version converter, if robn doesn't mind.

I'll read up on json first, then see if I can figure out the existing code. I'll probably then need to ask some questions.

Glad to be of help :)

Nick

[nick]

Hi again,

I've had a brief look at json - a bit like XML, but more readable.
To my understanding, json files are text, which means people could cheat far too easily :) Maybe a text file could be encrypted/decrypted on save/load?
[edit] Had a thought about json: perhaps robn intended json not to be the text format of save files, but as an intermediate state, i.e. all the objects will write to a json string instead of a Serializer writer object - then when done, the json string (containing the complete savegame state) can be used to create a structured file?

I've also had a look at the source code, in particular the save/load/serialize/unserialize functions. I've traced through function calls for a "load game" scenario:

Code: Select all

static int l_game_load_game(lua_State *l)
  Game *Game::LoadGame(const std::string &filename)
    Game::Game(Serializer::Reader &rd)
      // read galaxy generator
      // read game state
      // read space, all the bodies and things
      Space::Space(Game *game, Serializer::Reader &rd, double at_time)
        Body *Body::Unserialize(Serializer::Reader &_rd, Space *space)
          void Body::Load(Serializer::Reader &rd, Space *space)
            // call Serializer::Reader methods
      // read more data

It's going to take me a while to get my head around the code :)
My plan is first to understand the existing save/load, then assess what needs to be done and discuss the options (e.g. json or binary file with header). If I get that far then I should be in good shape to take out a branch and start coding :)

Nick

[jpab]

I've had a brief look at json - a bit like XML, but more readable.
To my understanding, json files are text, which means people could cheat far too easily :) Maybe a text file could be encrypted/decrypted on save/load?

It's trivial to cheat anyway -- anyone can open up the in-game script console and give themselves extra money or equipment or a better ship, etc (almost essential for development purposes). I'm happy for players to cheat if that's what they want to do; it's not a multiplayer game, so cheating doesn't affect anyone but the cheater, and I'd rather not try to dictate to people how they should be enjoying the game. That's just my opinion though. I don't know how other people feel about it.

It might be worth putting the JSON through a compressor (maybe LZ4 or something similar) just to reduce the amount of disk traffic, but that's an extremely minor detail that should be trivial to change later. During development it will certainly be best if the save files are plain text because it'll make debugging much easier.

A JSON based save system is going to be a fairly big project to do right. It looks like you're taking a good approach -- understanding things first is always good. Good luck and please do keep us updated on your ideas, and ask questions!

John B

[FluffyFreak]

I'm impressed :)

Yeah don't worry about encryption/decryption. If we want to then it can be something we add later at the file write/read stages.

Glad to see you're working on it though.

Andy

[nick]

Thanks guys :)

A readable JSON text file will be the nicest solution, I'm happy to go with that (I didn't realise there was an in-game script editor!)

Nick