Validating Coordinates

[wickedliving]

Hey everyone,

I'm looking for better alternatives (to save memory) for validating map coordinates. Currently, I load every dmap with xMax and yMax with a map[Vector2]Tile

This means I can lookup points using the the coordinates and the the valid, surface, and height of each tile.

This approach requires 100mb for Canyon alone, which I'm attempting to reduce it.

Any and all help would be appreciated!

[Spirited]

Yeah, for sure - loading game maps with tile information can be pretty memory intensive. I've seen a few approaches, like unloading maps entirely / or unloading quadrants / chunks of maps that aren't in use (like what Minecraft does). That can be helpful for lots of lesser used maps or portions of maps that are never loaded. But ultimately, game servers are just memory intensive. It's a big reason why huge MMORPGs like World of Warcraft or Guild Wars 2 have zones that players can "phase into" or "load to" via a loading screen (really transferring you to new game servers).

Just spit balling ideas, but you could make loading elevation optional. Aka. if a map is completely flat, then disable all height checks and load in a simpler tile definition (maybe make that an interface for tile flags and extend that interface for tile elevation). What you definitely want to avoid is memory compression techniques. I've seen that... and the performance is abysmal. What's your current tile definition?

[wickedliving]

I can't imagine the map sizes of the MMORPG's like GW2 and WoW. I thought about doing a Map struct and breaking it to quadrants like you said.

I've managed to make it around 65mb for canyon by ridding myself of the surface type and making my Vector2 coords uint16 instead of uint32.

Here are my current structs

type DMap struct {
Id    int
xMax  uint32
yMax  uint32
Tiles map[Vector2]Tile
}

type Tile struct {
Vector2
Valid   bool
Height  byte
}

type Vector2 struct {
X uint16
Y uint16
}

[Diab]

most large maps are around 900x900 and so 900 x 900 x 8 bytes is around 6.5 mbs.

[Spirited]

Ahh, yeah. The surface type is a cool addition I wanted to get into as well, but it definitely takes up a lot of memory. Hmm... so I don't think your tile definition needs the X and Y coordinate. It looks like your array is already indexed on X and Y. Removing those would free up some space. Also you could change your Valid boolean to a byte for a tile bitflag system at no extra space cost, which would allow detecting if a monster or item has already been placed at a coordinate (totally optional). Your height might be problematic since some maps go well into the thousands for height. I suppose changing that to a short could be optional though as long as you're ceiling your values so they don't wrap around on overflow.

You have to be a bit careful with how you reduce memory consumption, overall. In a lot of cases, it's much more performant to rely on in-memory caches and bitmaps (such as the case for using map tile flags for O(1) lookups rather than performing collection scans for at worst O(n) floor items / spawned monsters).

[leorbmello]

I don't know if it will help you, but... Like Spirited said, it saved me alot of memory usage. I made the server load basically all maps patch (GameMap.dat) and load just a few maps... most important ones, like city maps and quest maps. the most used ones, they're just a few, so.... The others if they would be used, the system loads the map when someone teleport there (yeah, you may feel like laggy when teleport to it for first time, but take no more than 2 or 3 seconds to the server load the map) and if the map has monsters or generator, the generator start just when the map has been activated.

[Rezlind]

I don't know if it will help you, but... Like Spirited said, it saved me alot of memory usage. I made the server load basically all maps patch (GameMap.dat) and load just a few maps... most important ones, like city maps and quest maps. the most used ones, they're just a few, so.... The others if they would be used, the system loads the map when someone teleport there (yeah, you may feel like laggy when teleport to it for first time, but take no more than 2 or 3 seconds to the server load the map) and if the map has monsters or generator, the generator start just when the map has been activated.

Just wondering if there is any benefit to "winding" down these maps if they had no activity for a while? I.e. turning off generators for those specific maps and trying to get it to unload from memory? Is that instead too risky to do once it's already loaded and thus better to just hold the high memory usage?

I recognize that maintenance sorts these issues out but figured I'd ask.

Thanks!

[wickedliving]

I was able to reduce the size pretty significantly by removing the unnecessary fields, including the extra Vectors that were already being used as the keys. I may actually create some sort of option flags for this dmap system since it will almost certainly be used in my stress test bot as well. I'm using this for more of a validation for player movements, item placement, and mob movements.

My project structure is a little different than other servers as I'm not taking the traditional map route. I've actually created my server around Zones, and each zone has a conquer map assigned to it. For instance, Twin City map id is 1002, but it's zone ID is 1. If I wanted to duplicate Twin City for single player instances (like GW1) I can do that. I plan on using this feature mostly for quests and guild/team instances. So my map system has to be reusable and completely separate from zones. I'm using the map points as a reference to make sure that whatever a player (or the server) is attempting to do is also valid with the client. Everything else, such as items on ground or mobs walking, is based on the zone and the entities inside of it.

Kind of off topic, but gives you an idea of why memory matters to me.

I really appreciate everyones help! Hopefully one day it's in a stable enough condition for me to release it.

[Spirited]

Hm. So I think I understand what you're going for, but I'm a bit confused as to why you need a separate identifier for zones. Conquer Online already supports a static map id and dynamic map id, which means you should be able to spin up as many duplicate instances as you want without additional identifiers. If you want to partition players to existing instances of a map like how GW2 does it (sorry I'm not familiar with GW1), then that's still keyed by static map id. You just might need the value to be an array of maps rather than a single map (or have that as a separate cache). Then just have an algorithm for selecting a map instance like round robin, lowest capacity first, or player id / assigned partition id.

I guess this brings up a larger question though: are you looking to completely duplicate a map or phase players in-and-out of content within small areas of a map? If it's the later, then you might benefit from having instanced map entity collections rather than instanced maps as a whole. That way, you're not completely duplicating floor grids over and over again. I suppose you could make this as complicated as you need it to be; but in its most basic form, you could index entities by a map progression id that defaults to zero. WoW calls this technique "phasing", which they use all the time for quests.

Edit: Just a side note, I hope you don't feel like you owe anyone in this community anything. Please don't feel obligated to release any of your work if you don't really want to. In this community especially, I want to make it very clear that all levels of programming are welcome. You have nobody to impress and nothing to prove. There are no "elites" here.

[leorbmello]

I don't know if it will help you, but... Like Spirited said, it saved me alot of memory usage. I made the server load basically all maps patch (GameMap.dat) and load just a few maps... most important ones, like city maps and quest maps. the most used ones, they're just a few, so.... The others if they would be used, the system loads the map when someone teleport there (yeah, you may feel like laggy when teleport to it for first time, but take no more than 2 or 3 seconds to the server load the map) and if the map has monsters or generator, the generator start just when the map has been activated.

Just wondering if there is any benefit to "winding" down these maps if they had no activity for a while? I.e. turning off generators for those specific maps and trying to get it to unload from memory? Is that instead too risky to do once it's already loaded and thus better to just hold the high memory usage?

I recognize that maintenance sorts these issues out but figured I'd ask.

Thanks!

I believe I may have been wrong about what I really meant ...

I do not remove what has already been created from monsters on the map, I just stop processing the respawn until a player returns to that map, avoiding creating entities that will not be active (this does not delete or remove the monsters that have already been created) previously)

As for the loading of the map, I do not see any problem for now in working in this way, since important maps are always loaded. What stops being processed is just the generator.

It was a simple idea, it may not be good in the long run, for now it is working well and without complaints, it is just a way to reduce the use of Generator'sThread.

It would be pretty bad if i had to be reprocessing and recreate every entity on the map every time it's been out of use.

Also, i guess for dynamaps, you could create something like:

ConcurrentDictionary> Objects

main index as MAP id and second entityID and role object... (stupid idea) this way you could use a single title base to hold some event maps as arena, you would not need to recreate the entire floor...