Canyon Suite: A Comet project (for 6192)

[Konichu]

I added support for MySQL on the Login Server so people don't need APIs to run the server.

I also tried to record a video yesterday to show how to run the server on Visual Studio and test it, but my dog attacked my cat and everything went to shit since I had to run to save the cat, soooo, since I am no video editor I'll not edit what I've done and I'll try to do it again later.

[Tkblackbelt]

Nice! Was that to avoid needing the rpc calls for the auth->game server handover? Curious what was making that difficult to work with?

Im using comet as one of my reference for my source and was planning to just use a rest endpoint for that part.

Also I hope your cat is ok! I have two dogs myself haha.

[Spirited]

Nice! Was that to avoid needing the rpc calls for the auth->game server handover? Curious what was making that difficult to work with?

Im using comet as one of my reference for my source and was planning to just use a rest endpoint for that part.

Also I hope your cat is ok! I have two dogs myself haha.

I'll let Konichu talk about their design, but I can speak to Comet at least. In an ideal setup, there'd be some way of service discovery. So like, whether the RPC is through a REST API or just StreamJsonRpc over TCP (like it is in the original Comet project), there'd be an endpoint you can hit to discover running services. Consul is a really good method for doing that. There're other clever ways of transferring auth to the game server, like through the database. As long as you keep TTLs in mind for the auth token you generate and keep up the record. Adding a distributed cache like Redis would also be good for token TTL.

[CptSky]

Was that to avoid needing the rpc calls for the auth->game server handover? Curious what was making that difficult to work with?

From my experience, the hard part (or badly written part) for the handover is:

- not having to share the account database with the game database

- not having the account server knows all game servers (and needing to be refreshed when one game server changes)

- not leaking the database IDs (or having a too deterministic login)

- not having a weak token that can be spoofed

- having the proper flow for full/busy states per game servers

-----

On my current in-progress source, I'm using a pub-sub pattern (with MQTT) to do the Auth <-> Game Server communications.

The Auth Server is a MQTT hub, and Game Servers connect to it so it lazily discover them (no need to have configs about Game Servers). Then, the Game Servers create a topic that the Auth Server is subscribed to with a wildcard (using a local MQTT client), and it periodically broadcasts its status (name, IP endpoint, number of players online, max number of players, degraded state). This gives info about whether it is full, and what to return to the players that try to connect.

Tokens are randomly generated, and sent to the right Game Server by publishing on a per-Game Server topic. The token is accompanied with the account ID and the expiration, so that info does not leak to the clients. And then, it is only a matter of having a memory cache to track all recently received tokens. It must be noted that there can be race conditions, and the player may connect to the Game Server too early. So I have a MsgConnectDelayed (custom message) that I enqueue if that happens with the original MsgConnect data, plus a timeout. And will try to re-process it until it expires (at which point, I assume that I'll never receive the token from the Auth Server).

[Tkblackbelt]

Was that to avoid needing the rpc calls for the auth->game server handover? Curious what was making that difficult to work with?

From my experience, the hard part (or badly written part) for the handover is:

- not having to share the account database with the game database

- not having the account server knows all game servers (and needing to be refreshed when one game server changes)

- not leaking the database IDs (or having a too deterministic login)

- not having a weak token that can be spoofed

- having the proper flow for full/busy states per game servers

-----

On my current in-progress source, I'm using a pub-sub pattern (with MQTT) to do the Auth <-> Game Server communications.

The Auth Server is a MQTT hub, and Game Servers connect to it so it lazily discover them (no need to have configs about Game Servers). Then, the Game Servers create a topic that the Auth Server is subscribed to with a wildcard (using a local MQTT client), and it periodically broadcasts its status (name, IP endpoint, number of players online, max number of players, degraded state). This gives info about whether it is full, and what to return to the players that try to connect.

Tokens are randomly generated, and sent to the right Game Server by publishing on a per-Game Server topic. The token is accompanied with the account ID and the expiration, so that info does not leak to the clients. And then, it is only a matter of having a memory cache to track all recently received tokens. It must be noted that there can be race conditions, and the player may connect to the Game Server too early. So I have a MsgConnectDelayed (custom message) that I enqueue if that happens with the original MsgConnect data, plus a timeout. And will try to re-process it until it expires (at which point, I assume that I'll never receive the token from the Auth Server).

Nice, that's a good way to do it! Guess using a messaging queue you can make it easy to apply back pressure. The occasional broadcasting of the stats is a nice idea. Can then reject connections on the auth server if the game server is busy, without needing to fetch that info on every client connection.

I'm working on the same part of my server at the moment lol. I'm using Netflix Eureka as a service discovery layer. Then my auth and game server both register with it allowing them to know about each other. I'm just doing the auth-game "handover" using a rest call for now to keep it simple for now. I just have some throttling enabled + retry logic in place.

So far I'm really enjoying using Spring Boot + Kotlin for the server. I'm designing it to be work reactively using stuff like channels, coroutines, etc... Once I finish up this handover part, I'm going to figure out how to design the game server. Very excited for that part :)

[Konichu]

I've been away for a few time but I may talk about some decisions now xD (so much work was driving me mad, so I had to leave some things behind).

I actually did not leave REST, it works well for me (except when I forget to renew my SSL certificates and login crashes lol), I just did that change to the code so less people ask me "how to set up the source". Canyon of course has a few steps more than common sources but it's not impossible, there are people using it already.

I'll actually look into some recommendations you both gave above haha, I am building a prototype of Canyon which I called Long (very creative Konichu) and I am attempting new stuff on it, including a copy of TQ modules using DLLs

http://i.epvpimg.com/d4RJaab.png

http://i.epvpimg.com/fWYeaab.png

I actually did all the communication via socket because I just didn't like the design of a two way rpc (or my code was ugly idk), also it's was easy at the time to setup network security and etc.

About the APIs I thought it was a good idea to authenticate and get stuff from them because after seting up contracts all I need to do is change the APIs code, it was good to fix bugs in the ban and vip system without needing to restart anything from the server.

I'm also studying and trying to implement C3 on OpenGL to maybe bring to the community a starter pack for a new Conquer client.

Edited November 21, 2023 by Konichu

[Baloony]

Will you be willing to sell the full source with login server and the full db? Btw thank you for all your hard work!

[Tkblackbelt]

Will you be willing to sell the full source with login server and the full db? Btw thank you for all your hard work!

This is an open source project... The code is literally in the first post lol.

[Konichu]

Will you be willing to sell the full source with login server and the full db? Btw thank you for all your hard work!

DB is not for sale.

[Baloony]

It seems the database strucutre has changed since you last updated the batch, would you mind uploading a more current one, of course without cq_npc and cq_action?

[Konichu]

I think that this will have the missing structures.

cq_6135_full_structure.rar

[Baloony]

Thank you so much man!

I tried connecting with a 6192 client but although it worked and I got logged in, jumping seems bugged, my question is, which is the real patch that the packets are most compatible with

[Konichu]

I actually develop on a retail 6192 client...

[Baloony]

Could it be from the missing actions and tasks? Does jumping work fine for you, also attacking makes the mob disappear until i move again, are theese known bugs?

I would love to contribute

[Konichu]

Could it be from the missing actions and tasks? Does jumping work fine for you, also attacking makes the mob disappear until i move again, are theese known bugs?

I would love to contribute

No, Canyon has been used in a live server that reached over 180 concurrent accounts. But I may take a look into it soon, I've been working on some stuff lately and haven't been developing Canyon.

[Baloony]

My bad, my character's id was 4 and that made the source think i'm an Armorer npc, all looking good so far man, you are trully one of the greatest!

I set all caharacters ids to be 10kk + and it all works fine now.

[Baloony]

Do you have any idea why ArtisanCloud(GemInsetter) and ArtisanWind would not open their gui when clicked?

Also, you mentioned something about a leaked 6900 database, can you point me to where or how I can get it please?

[Konichu]

I'll PM you later with a 7117 LUA and database.

[Baloony]

That would be great man, I would really appreciate it!

[Alisson]

Hello, I needed help, I left the GameServer and LoginServer online, registered a new account in the database using hash, configured the client, tried to log in, used an account and put a different password, with the intention of observing the login server , and as expected the incorrect password error appeared, after that I tried to log in again, using the correct password the login server didn't give any message, but in the game client it stuck on the ''Logging in the account'' login screen, like Before you gave the wrong password, I can discard the client's part, right? the configuration is correct.

[kennylovecode]

Hello my friend , when i start the project .... game server can't connect login server , Because the database does not have this “Realm” record.... How can i get this record .... or i can insert a record to fixed this problem?

[Konichu]

Hello, I needed help, I left the GameServer and LoginServer online, registered a new account in the database using hash, configured the client, tried to log in, used an account and put a different password, with the intention of observing the login server , and as expected the incorrect password error appeared, after that I tried to log in again, using the correct password the login server didn't give any message, but in the game client it stuck on the ''Logging in the account'' login screen, like Before you gave the wrong password, I can discard the client's part, right? the configuration is correct.

You need to debug this yourself, check if the player data is being transferred between both servers, which packet has been sent/recv last and track if it was a login, game or network issue. And also make sure that your realm is properly set up on your database.

Hello my friend , when i start the project .... game server can't connect login server , Because the database does not have this “Realm” record.... How can i get this record .... or i can insert a record to fixed this problem?

You need to register your new realm, if you need to see how it's done just take the Piglet repository and see the realm management page.

[Alisson]

So I think I figured it out, I didn't configure the api part of the accs, I'll try to remove it and add it via mysql.

[Konichu]

So I think I figured it out, I didn't configure the api part of the accs, I'll try to remove it and add it via mysql.

Just define the preprocessor variable USE_MYSQL_DB

https://gitlab.com/world-conquer-online/canyon/canyon/-/blob/main/src/Canyon.Login/Sockets/Login/Packets/MsgAccount.cs?ref_type=heads#L21

Just add a try/catch block to avoid logins getting stuck if DB is not online

https://gitlab.com/world-conquer-online/canyon/canyon/-/blob/main/src/Canyon.Login/Repositories/AccountRepository.cs?ref_type=heads#L7

[kennylovecode]

The first , thanks for shared ！

I believe it's driven by a spirit of dedication

But when I cloned the code repository locally, I found that I wanted to compile the source code and run it. . . This requires me to have strong code reading skills

Moreover, the database is not complete, and project files often have missing configuration items.

For example, when I tried to run the Canyon.GM.Panel project, even the most basic database link string was not included in the configuration items. I needed to compare them one by one and improve them.

Then you have to modify the warehousing mapping, and the table name does not correspond. . . .

Maybe I need to spend a lot of time on this, and there is no guidance document. . .

But it's still cool. This is best one of the `Conquer Online` open source projects I've seen that has a clearer structure and is easier to manage later.

And the structure of the database is also very clear, basically consistent with the official library, which surprised me. Unfortunately, there is no data reference.

Of course, this is also thanks to "Fang"'s open source contribution to the infrastructure project Comet.

Hope this gets better and better. . . .