XComp's Forums

A place to discuss the mods of XComp
It is currently Mon Jul 24, 2017 2:40 pm

All times are UTC




Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Where to go from baselining
PostPosted: Sat Jun 27, 2015 11:22 pm 
Offline
Mystcraft Developer
User avatar

Joined: Sat Aug 25, 2012 9:33 pm
Posts: 3964
(Taken from the blog post)
Sometime back (I can't even place when) I was playing on FC2 and some stuff happened. I had a nice little house on the beach near the giant wand recharging tower, if you want to try and place it.
The event in question was incited by cpw, who created a Mystcraft age. This age had diamond ore tendrils.
While we were merrily collecting as many diamonds as we could carry in a world falling apart around us, cpw remarked, "So, XComp, how broken is your mod?"
I responded, "I'll fix it." I must have been channeling my inner Bill Cosby (Soaryn will get that joke).

At the time, instability was estimated by the symbols that did the generation using formulas provided by the modifiers they were using. This was really hard to balance in any kind of logical way and came with a large set of issues like the prediction being possibly wrong in either direction. I couldn't simply make diamonds cause more instability, either, as that'd make the other issues worse.

Instead, I came up with a system for profiling the world as it really was, not how it might be. I built the age profiling system. This is not the baseline profiling system, which I'll call baselining from now on (I'll try, anyway).
The age profiling system is pretty nifty, but difficult to explain to people who don't know code, so I'll try it with a Minecraft imagery metaphore.
Imagine you have a world with a bunch of chunks in it. Normally we can't see the chunks, but we know they are these 16x16 areas that load and unload and generate when needed.
Now, imagine all of those chunks slightly transparent. As if we'd made them holograms.
Now overlap them on top of each other.

It'd look a little weird, perhaps, but we'd quickly be able to see areas which were <em>usually</em> stone, or usually air, or might have trees. If we did this with thousands of chunks, we'd be able to see what range of blocks the diamond blocks usually show up in.

Mystcraft's profiling system basically does that. It goes through every chunk we generate and checks each block in it, mapping it to our single transparency. It actually counts the number of times this relative cell was a solid block, or an ore block, or water, or whatever else we want to count.
This allows Mystcraft to say, "You know, this world has a lot more diamond than normal..." and the produce instability because of it, in a nice, logical way that's actually pretty easy to balance.

The catch was, "what's normal?"
About a week later I started playing on Forgecraft again. This was actually the beginning of the last time I played on FC2, I think. The one where we used the configs from the DW20 pack...
Turns out, someone had played with the vanilla ore distribution. Since Mystcraft was comparing the profiling data to vanilla's normal ore distribution, it wound up making all ages on the server unstable. No one had bothered to test this in advance, so the DW20 pack users were all complaining they couldn't get stable ages...

And so, I was tasked with fixing this too.
Now, because the profiling data was pretty complicated (and for a few other reasons besides) I determined to simply establish what was normal for the save and use that. Thus the baselining was born.
The baselining usues the profiling system, but it is actually more than that. What it does is it generates an extra dimension in the background; a pure dimension which we can determine is "stable."
As the dimension generates, it gets profiled, same as any other Mystcraft age. We can then use this data to do the comparisons for "this world has a lot more diamond than normal" without worrying about the mod loadout and configs.

And then, there were more problems.
The baselining was actually pretty slow at first. I made a valiant effort and improved lots of things and had it running really smoothly for the most part, but I still got complaints. Apparently my four year old laptop still outpaces the standard hardware... Cool, I guess.
At this point the baselining runs pretty fast, so long as mods play nicely. Many mods that do generation don't seem to understand how generation should work, and so they do things which make generation take many times longer than it should, but it still sorta seems to work.

It's not enough. The top complaint I hear is "why does it happen for every world?!"
The two main reasons for this were that it made it possible to do the baselining in the background while you played and that some mods actually allow you to change the ore distributions per save...

I gave up and tried something new. Something experimental and something I don't even like: I moved the profiling to happen when you start the game.
I'll be clear on this: I hate this idea.

I've liked it even less since I released it. This system is different enough that it confuses too many mods. There is a crash on game start with bspkrsCore (he's aware of the issue and accepted the fault) and sometimes the progress bar just doesn't move. This means no chunks are generating. I have no idea why that is. Some mod is doing something, but that's nothing to go on.

And now that's the issue: the wait; it runs when things change; the conflicts with other mods.

Apologies if my tone comes across as irritated, but I'm tired; really tired; of this issue. Mystcraft was more fun when I could make it about the lore and not have to try and balance the darn thing.

With that explained, I'm currently trying to get feedback on how to move forward with it. I welcome your suggestions.

_________________
I am the Mystcraft Dev.


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sat Jun 27, 2015 11:40 pm 
Offline
Poster

Joined: Tue Aug 27, 2013 8:54 pm
Posts: 1
Instead of creating a new dimension wouldn't it be possible to do the baselining on the overworld and then store it? Overworld would still be influenced by these vanilla ore generation tweaks wouldn't it?


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sat Jun 27, 2015 11:41 pm 
Offline
Mystcraft Developer
User avatar

Joined: Sat Aug 25, 2012 9:33 pm
Posts: 3964
It's not quite the same, unfortunately. A good suggestion though. :)

EDIT: My current suggestion is to add these config options:
baselining.useconfigs=false // If true, the baseline calculations won't run and instead a config file will be read.
baselining.per-save=false // If false, the profiling will run on game startup with the loading bar. If true, it will run in the background when playing.

_________________
I am the Mystcraft Dev.


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 12:10 am 
Offline
Poster
User avatar

Joined: Sat Feb 02, 2013 6:07 pm
Posts: 17
Location: Wisconsin
I like the config idea, one thing I would ad is that when baselining.useconfigs=true is set and there is no config file it should generate one that reflects vanilla ore distribution.

_________________
I haven't lost my mind! I know it's around here somewhere...


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 12:39 am 
Offline
Wiki Admin

Joined: Thu Apr 03, 2014 10:55 pm
Posts: 715
I think the current system is making the best of a tricky task. And your proposed config file rules give a decent backstop for those who won't wait. I just started a new world with Forge 1428, that now has a graphical progress screen as it starts up and sorts out the mods -- the Mystcraft profiling does seem to fit nicely into that.

One additional option you could add, would be to run during play time (avoiding the front-page tsuris), but in the background at low priority so it doesn't lag the game. (Unless switched by the config, it would still build the per-profile config files.) This would suffice for a typical survival player, who probably isn't going to be leaping into an Age in, say, their first 20 minutes of play -- and by the time they do get iron armor and ready to head into Ages, it'll be done. If they do enter an age before the profiling's been done, they get an in-game warning that their world's instability may be unstable and overwrought due to interdimensional labor actions, then it falls back onto the vanilla config. When the profiling does finish, anybody actually in an Age gets, say, a five-minute warning until switchover, just in case they gain instability.


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 1:00 am 
Offline
Wiki Admin
User avatar

Joined: Tue Sep 18, 2012 5:27 pm
Posts: 3826
spiderbiggen wrote:
Instead of creating a new dimension wouldn't it be possible to do the baselining on the overworld and then store it? Overworld would still be influenced by these vanilla ore generation tweaks wouldn't it?


There's still an issue of ores-that-generate-in-specific-biomes. If you start a world and don't have that biome (cough, extreme hills, cough) then the baseline data will be wrong.

_________________
[WARNING: This user is sometimes blunt and confrontational. Please don't take it personally.]
[Current Mood: Agravated, Git:¹ Very high probability of being unintentionally hostile. I'm a nice guy, honest.]

¹http://xkcd.com/1597/


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 4:50 am 
Offline
Wiki Admin
User avatar

Joined: Mon Aug 27, 2012 7:42 pm
Posts: 4575
My thoughts:

Zero: Profiling what is actually generated, versus assuming a vanilla world, is a no-brainer. Equally, charging based on the actual age, instead of just the symbols, is also a no-brainer. Putting in some additional charges based on the symbols does, I think, make sense -- the more constraints you put into the world, the more likely there's some unintentional conflict.

First, we have some mod(s) out there that mess up the out-of-game profiling. So:

Have an option to select high-priority pre-play profiling versus low-priority during play profiling.

For the "low priority during play", give people the ability to turn its speed down. One chunk per tick may not sound like much, but for many people, 20 TPS is actually a challenge in normal play. And yes, since you are not likely to go to an age that quickly, letting mystcraft do one chunk every 5 ticks instead of every tick won't really cause mystcraft issues, but will help things work better.

Second, once you do have a profile, even if it was made in a world, instead of the pre-world, store it as the base pack's profile.

Third, have some sort of "full speed profiling" command for mystcraft, so that if you are doing something like creative mode testing, you can get that profiling done NOW!.

Fourth, support both a "pack default", and a "save specific" option -- allow some saves to have different configs and profiles. Most packs won't need this -- no server setup will need this. But some people will have different configs for different single player worlds. This one should be considered low priority / low demand / low need.

Quote:
Instead of creating a new dimension wouldn't it be possible to do the baselining on the overworld and then store it? Overworld would still be influenced by these vanilla ore generation tweaks wouldn't it?

You cannot assume that the overworld matches general ages. Imagine (this is real, by the way) the idea that the overworld is poor, and people go to ages for resources.

_________________
0.10.13.1-0.11.0 (and later): Safe linking and Mystcraft introduction: https://www.youtube.com/playlist?list=P ... HC6PDjh7ns

Currently a work in progress (3 of 5 parts finished). A 5 minute "TL; DW" summary is at the end of part 3. The first 10 minutes of part 3 are a summary of parts 1 and 2; part 3 is a decent job of being a self-contained mod spotlight, and does significant details on how to write worlds.

Part 4 will be looking at dangers of ages, coming soon


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 4:57 am 
Offline
Wiki Admin
User avatar

Joined: Mon Aug 27, 2012 7:42 pm
Posts: 4575
Oh! I missed this the first time reading:
Quote:
So make it so that the profiler doesn't do anything for like 30 seconds after a player generates a real world chunk. That way, if they're exploring, they're not profiling, but as soon as they stop to build a house for the night or dig up some exposed iron, the game races to get the profiling done.


How about tracking the TPS/time per tick on the server for a while, and try to determine what this pack/computer is able to do. Then, only run a profiling chunk if the current TPS is high / the current time per tick is low?

_________________
0.10.13.1-0.11.0 (and later): Safe linking and Mystcraft introduction: https://www.youtube.com/playlist?list=P ... HC6PDjh7ns

Currently a work in progress (3 of 5 parts finished). A 5 minute "TL; DW" summary is at the end of part 3. The first 10 minutes of part 3 are a summary of parts 1 and 2; part 3 is a decent job of being a self-contained mod spotlight, and does significant details on how to write worlds.

Part 4 will be looking at dangers of ages, coming soon


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 5:17 am 
Offline
Poster
User avatar

Joined: Sun Jun 28, 2015 5:11 am
Posts: 2
Location: Germany
I think the profiling at the start of the game is better because it is faster for me and when my server first profiled the world it took over hald an hour and the server had TPS arround 600ms. But for Modpacks the profiling at the beginning is realy bad because every user must run it even when they connect to a server that already did it. I play on my server with a 160+ mods and GregTech and so far we made no mysctarft ages but we also had no crashes so I think it is working with Gregtech.
The latest version of mystcraft is fast profiling. The version before loaded and unloaded the world arround about 3 minutes.


Top
 Profile  
 
 Post subject: Re: Where to go from baselining
PostPosted: Sun Jun 28, 2015 11:51 am 
Offline
Mystcraft Developer
User avatar

Joined: Sat Aug 25, 2012 9:33 pm
Posts: 3964
keybounce wrote:
How about tracking the TPS/time per tick on the server for a while, and try to determine what this pack/computer is able to do. Then, only run a profiling chunk if the current TPS is high / the current time per tick is low?

A rate limiter might be possible, but I won't make it a priority.

SuperSandro2000 wrote:
I think the profiling at the start of the game is better because it is faster for me and when my server first profiled the world it took over hald an hour and the server had TPS arround 600ms. But for Modpacks the profiling at the beginning is realy bad because every user must run it even when they connect to a server that already did it. I play on my server with a 160+ mods and GregTech and so far we made no mysctarft ages but we also had no crashes so I think it is working with Gregtech.
The latest version of mystcraft is fast profiling. The version before loaded and unloaded the world arround about 3 minutes.

Well, that's the nice thing about configs, you'll be able to do the way that works for you. :)
Note that the server already has an option to prevent people from connecting while the profiling is ongoing.

Actually, I went through the effort to make it so that you don't need to complete the profiling to play multiplayer. If you connect to a server then the profiling halts.

The loading/unloading world issue is some other mod deciding it should have the power to unload worlds based on its own heuristics. It's doing it very wrong, as the profiling dimension is marked as "keep loaded." It shouldn't go away until Mystcraft lets it. The loading and unloading can actually cause major issues and slow down the profiling dramatically. If you can track down which mod is causing the issue then we can report it to them and things will improve. :)

_________________
I am the Mystcraft Dev.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group  
Design By Poker Bandits