When I was still in middle school, my class would play a game called Shakes And Fidget. It is pretty easy to play, so of course there was a bot you could use to play the game for you. This bot is actually what got me my first experience with Linux, when I figured out how to run it on a Raspberry Pi 24/7.

Roughly a decade later and with a the experience required to make and not just use programs, I wanted to give back to the community and make something, that others could use and maybe get them interested in programming themselves.

To do anything useful, I would at least have to be able to send requests and parse the responses from the server. My first starting point to understand this communication was the bot I mentioned previously. Sadly it is not open source and their decompiled C# is heavily obfuscated.

Almost makes you think this is a malicious program, but from what I can tell, they actually just don't want anyone to read their code.

Well, this is a browser game, so let's just look at the requests the game is doing in the background.

Browser Requests

https://f8.sfgame.net/req.php?req= 0-00000000000000 iY0Ap3B2omAw8Cx-hkM-uVt5DjJgRqZHqWLBMPNmGmWh8M6_f9jsqTxvFMEZXT66KozZ98Xiu1aUxnqhP6cvIQ== &rnd=0.1250823&c=3
https://f8.sfgame.net/req.php?req= 0-00000000000000 iY0Ap3B2omAw8Cx-hkM-uVt5DjJgRqZHqWLBMPNmGmWh8M6_f9jsqTxvFMEZXT667pjQlolao5KESTCA9eE8Bw== &rnd=0.357428&c=4
https://f8.sfgame.net/req.php?req= 0-00000000000000 iY0Ap3B2omAw8Cx-hkM-uVt5DjJgRqZHqWLBMPNmGmWh8M6_f9jsqTxvFMEZXT66KozZ98Xiu1aUxnqhP6cvIQ== &rnd=0.07045985&c=5

These are the requests being sent to see, if the name for a new character is still available. You can immediately notice, that:

  • They do cache busting with a rnd parameter
  • There seems to be some base64 & suspicious zeroes
  • They count the amount of requests that have already been sent
  • The req parameter for 1 and 3 are identical (they are for the same username)

Base64-decoding the parameter yields:

‰§pv¢`0ð,~†C>¹[y2`F¦G©bÁ0ófe¡ðοØì©<oÁ]>º*ŒÙ÷Åâ»V”Æz¡?§/!

which suggests, that we are dealing with some sort of encryption, or at least obfuscation.

Let's see, if we can figure out more by registering an account. We do not understand the requests yet, so we just look at the response from the server:

cryptoid:  0-wfDawnfXmrUU1Y
cryptokey: 47sF686zf0Z6aOf6
sessionid: dWkT5mVOlTRrU1EgLzdAgQWABATVsHCz

This response has been reformatted and cut down to the important bits

Ok, so we get a cryptokey. I guess the request is indeed encrypted. All new (authenticated) requests now look something like this:

https://f8.sfgame.net/req.php?req= 0-wfDawnfXmrUU1Y dve6rnCfafdCaOyl8OdXirYlEnLjviA6IEnzXgWiWlYM1rgVMieB6N3_NYSK0VolUPKPD9XUdnHppnSteNTDow== &rnd=0.2173368&c=9

That gives us the following pseudo-code for the URL:

let req = cryptoid + base64enc(encrypt(request));
return (
  BASE_URL + "/req.php?req=" + req + "&rnd=" + random() + "&c=" + COUNTER++
);

That means we only need to figure out what encrypt() does to (hopefully) write our own decrpyt() function, that can tell us what the client is actually sending. Otherwise all variables here are known.

I have grown to love CyberChef for just messing around with unknown encryptions/obfuscations, but in this case nothing resulted in clear text. It seems like we are at the point, where we have to actually reverse-engineer the game.

I actually just randomly stumbled upon the required parameters here originally, but that is pretty anticlimactic. Instead I warp the timeline here a bit and put a later reverse-engineering attempt here

Game Files

Looking at the network requests again, we can see, that the site loads two big files, *.wasm.gz and a *.data.gz.

After (g)unzipping both, we are left with a WASM file and some form of data, that not even the file command understands. That means we need to look at the actual header:

> xxd data | head -n 3
00000000: 556e 6974 7957 6562 4461 7461 312e 3000  UnityWebData1.0.
00000010: 1201 0000 1201 0000 82ca ad00 0c00 0000  ................
00000020: 6461 7461 2e75 6e69 7479 3364 94cb ad00  data.unity3d....

Seems like we have a UnityWebData1.0 file... a few google searches later I had the appropriate tool ready and extracted the contents:

> uwdtool -u -i data
> ll output
boot.config
data.unity3d
Il2CppData
Resources
RuntimeInitializeOnLoads.json
ScriptingAssemblies.json
sharedassets0.resource

Looking through the files, it became clear, that another level of unpacking was required. The best tool for the job seemed to be Il2CppDumper.

> mkdir output_cpp
> Il2CppDumper wasm output/Il2CppData/Metadata/global-metadata.dat output_cpp
> ll output_cpp
common
DummyDll
dump.cs
il2cpp.h
script.json
stringliteral.json

Great, we actually have a readable dump.cs with code and stringliteral.json with all the strings we ever need.

If you want to look at these files yourself, you can get them here

Skimming through the code, I found a very interesting part:

public abstract class ARequest : ACommand<GameModel>, IRequest
{
    private const char Separator = '\x7c'; // '|'
    private const string CryptoPrefixNoAes = "0-000000000000-0";
    private const string NoSessionID = "00000000000000000000000000000000";
    private const string RequestPrefix = "/req.php?req=";
    private static readonly string CryptoPrefixAesDefaultKey;

For reasons unknown to me, but probably related to the static keyword, the last assignment is missing. Despite this, we can tell from the variable name, that we are using AES encryption for the requests. AES encryption needs a key and an IV (initialization vector). The key (at least for authenticated messages) is known from the server response, but we still need the IV. Searching the code for variables, that are called something along these lines, I could not find one, that made sense. We do however know, that the initialization vector has to be a string somewhere and there is conveniently the stringliteral.json file with all strings right besides the code. It contains ~20k values, so we need to filter these a bit. AES has different "flavours", (128bit, 256bit, ..) and each of them needs an initialization vector and key, that are that exact size. Since we know the key the server has send us back (47sF686zf0Z6aOf6) is 16 bytes/characters, we can assume, that we are using AES128 and only need to search for an IV, that is that length.

> jq -r '.[].value | select(length == 16)' stringliteral.json > "16bytes.txt"
> wc -l 16bytes.txt
     492 16bytes.txt

That leaves us with just ~500 strings. My original idea was to write a tool, that tests all of them and then search the output for strings, that would make sense in a request. Most of the values were just english text however, so I just manually removed all values, that were obviously just variable names, or game text. This narrowed down the list to just three:

  • 363VmcXgU11x8xtN
  • [_/$VV&*Qg&)r?~g
  • jXT#/vz]3]5X7Jl\

That is an amount, that I can just manually plug into CyberChef again and guess what. It worked:

That means we know we are using AES128 CBC (no padding) and an IV of jXT#/vz]3]5X7Jl\.

The only thing left is figuring out what the cryptokey is for requests, that are not logged in. Since we know they are 16 bytes, it must just be one of the two remaining values. Turns out it is [_/$VV&*Qg&)r?~g link

Great! Since AES uses the same parameters for encryption and decryption, we can now AES decrypt and encrypt every request. Let`s take a look at how the decrypted requests are actually structured.

Raw requests

Looking back at the C# code and an unauthenticated request:

private const char Separator = '\x7c'; // '|'
private const string NoSessionID = "00000000000000000000000000000000";

00000000000000000000000000000000|AccountLogin:brukond/c2de194903df6de2d02eaa3349c54e36481368c5/1/unity3d_webglplayer//225020000000///0||||||||||

We can assume a (not yet encrypted) request is roughly build like this:

let params = params.join('/');
let result = sessionid + '|' commandname + ':' + params;
while length(result) % 16 != 0 {
  result += '|'
}
return result

The sessionid is known after the login and defaults to zeroes initially. That means the only unknown are the command names and their parameters. We can actually just look back at the cs code to see all available command names

public static class RequestNames
{
    public const string AccountLogout = "AccountLogout";
    public const string AccountLogin = "AccountLogin";
    public const string AccountDelete = "AccountDelete";
    public const string AccountCreate = "AccountCreate";
    ...

The parameters are a bit difficult to find this way though. We can get a rough idea of them, by looking at their definitions and see which fields they have, but the order and concrete values are up for us to figure out.

public class AccountLoginRequest : ARequest
{
	public string Name { get; set; }
	public string Password { get; set; }
	public string HashedPassword { get; set; }
	public long LoginCount { get; set; }
	public string DeviceName { get; set; }
	public string DeviceToken { get; set; }
	public long ClientVersion { get; set; }
	public int OldGameWorldID { get; set; }

Just to get us started, we should probably look at the login request, as it would be always required to interact with the server. Just by the parameter names and the request we have looked at before, we can tell that the password is going to be hashed. Looking back at the code, we can find the following hints:

public static class RequestHelper
{
  private const string PasswordSalt = "ahHoj2woo1eeChiech6ohphoB7Aithoh";
  ...
  public static string GetSha1Hash(string text, Encoding encoding) { }
  public static string HashPasswordSaltOnly(string password) { }
  public static string HashPassword(string password, long loginCount, Encoding encoding) { }

We can see, that:

  • We use sha1 hashing
  • The salt (a value to change the hash a bit) is ahHoj2woo1eeChiech6ohphoB7Aithoh
  • We have two hashing functions.
    • One that uses the salt
    • And one that takes in a loginCount parameter

We don't know the value of loginCount, but can guess, that it is 0 or 1, since the client has no other information about the amount of times someone has logged in. Otherwise, we should have all we need to experiment again. Since hashing is not recoverable, we can not "decrypt" the hashes. Instead, we do a "known plain text attack". Since we can just try to login with any password, decrypt the request and see the produced hash, we try to recreate the hash from the same input.

Trying out the password 123456, we get the hash 705d0cc8c54efe7a040f7872507bd9fc4bd23bbf. We could now write a program, that tries out all combinations of concatenations between password, loginCount and salt. Once again though, moving a few blocks in CyberChef turned out to just work

Noting these steps as code, we get:

let hash1 = sha1(password + "ahHoj2woo1eeChiech6ohphoB7Aithoh");
return sha1(hash1 + "1");

Now we can actually login a character. The only thing missing, before we can fully replace the client is... we need to figure out how to login an account. Yes, even though we figured out login for characters, there is another layer above that. Accounts were introduced a few years ago and have multiple characters below them. Characters linked to an account can not be logged in via normal credentials and must be obtained through another API, only accessible with a logged in account. The parsing is very straight forward, no encryption, no reverse engineering, just looking at the requests and interpreting them. If you actually care, you can take a look at this chapter by clicking on "Bonus Chapter", but it is not "required reading".

Bonus Chapter: SSO Accounts

SSO Accounts

An accounts has 4 different possible ways of logging in:

  • Email (username & password)
  • Steam
  • Facebook
  • Google

The last three are SSO-providers, which means they handle the authentication part. For now though, we should look at the simpler email method in the browser requests first.

POST: https://sso.playa-games.com/json/login?client_id=i43nwwnmfc5tced4jtuk4auuygqghud2yopx&auth_type=access_token
PARAMS:
  - username: testuser
  - password: 42366516092a57cbf5e82ebd32c8beecaf750672
  - language: en

Instead of weirdly encrypted, padded and encoded GET requests, we have a sensible looking POST request. You might notice the client_id parameter in the URL, but this seems to just be a constant and can be ignored. The only interesting part is the hash, as it is not the same as before, despite using the same 123456 password. This time though, there no need for a new reverse-engineering attempt, as I had already seen this hash before, when tinkering around with CyberChef. This it the hash, that you get, if you just add a "0" instead of a "1" in the second round of hashing.

let hash1 = sha1(password + "ahHoj2woo1eeChiech6ohphoB7Aithoh");
return sha1(hash1 + "0");

With that done, we could now send valid requests to the server ourselves. Let's look at what the server sends us back for valid credentials.

{
  "success": true,
  "status": 200,
  "data": {
    "token": {
      "token_type": "Bearer",
      "expires_in": 28800,
      "access_token": "eyJ0eXAiOiJKV1QiL..."
    },
    "account": {
      "uuid": "04d70e90-36c4-4394-bae1-38bf95fb7397",
      "username": "testuser",
      "email": "z********@g******.***",
      "email_payment": null,
      "confirmed": true,
      "credential_login": true,
      "language": "en"
    },
    "sso": []
  }
}

If you have ever looked at web auth, this should look very familiar. What this request is telling you is, that you should add a Authorization: Bearer <access_token> header to all new request. This signals to the server, that the request is authorize from that account. The response also provides more details for the account like the username (important if you logged in via an SSO provider) and a `UUID'.

After successfully logging in, the server is also sending another request.

https://sso.playa-games.com/json/client/characters

{
   "success":true,
   "status":200,
   "user_id":387124,
   "data":{
      "characters":[
         {
            "id":"490477c033a07be5842d375b095dcb02",
            "name":"testcharacter",
            "server_id":472,
            "char_class":1,
             ...
         }
      ],
      "servers":[
         310
      ]
   }
}

The only tricky part her is that we only have a server_id, but we need a server URL to send further requests. Luckily the lookup table for that can just be found here: https://sfgame.net/config.json:

{
  ..
   "servers":[
      {
         "i":472,
         "d":"s6.sfgame.eu",
         "c":"eu"
      },
      {
         "i":475,
         "d":"s7.sfgame.eu",
         "c":"eu"
      },
      ...
   ]
}

So, server_id: 472 means we should send requests for that character to s6.sfgame.eu. Simple enough, let's look at how the character is going to be logged in now, since we do not seem to have a password per character. Skipping over the decrypting part, this is the request send to the server by the client:

SFAccountCharLogin:04d70e90-36c4-4394-bae1-38bf95fb7397/490477c033a07be5842d375b095dcb02/unity3d_webglplayer//225020000000

Replacing the known variables we get:

SFAccountCharLogin:{uuid}/{char_id}/unity3d_webglplayer//225020000000

Looks like we have everything. This request is functionally the same as the normal login. The only thing missing is support for the other providers. Thankfully, these are very similar and also straight forward. We just request https://sso.playa-games.com/json/sso/{provider_name}, which gives us

{
  "success": true,
  "status": 200,
  "data": {
    "redirect": "https://VERY_LONG_AUTH_URL",
    "id": "b0a56f6b70fd9a9f428aa2392a7bee91"
  }
}

Our user (us for now) then opens the redirect URL and uses it to login. To check if it worked, we query https://sso.playa-games.com/json/sso/{providername}/check/{id_from_last_response} and wait for a success response like this:

{
  "success": true,
  "status": 200,
  "data": {
    "access_token": "BEARER_TOKEN",
    "expires_in": 3599,
    "scope": "URL",
    "token_type": "Bearer",
    "id_token": "ID_TOKEN",
    "created": 1730458171
  }
}

Using the provided BEARER_TOKEN, the client then sends a:

POST: https://sso.playa-games.com/json/login/sso/googleauth?client_id=i43nwwnmfc5tced4jtuk4auuygqghud2yopx&auth_type=access_token
PARAMS:
  token: "ID_TOKEN"
  language: "en"

and receives back the full information about the account:

{
  "success": true,
  "status": 200,
  "data": {
    "token": {
      "token_type": "Bearer",
      "expires_in": 28800,
      "access_token": "BEARER_TOKEN"
    },
    "account": {
      "uuid": "15c59b59-9884-4c5a-ab8a-9387dc21644e",
      "username": "testuser",
      "email": null,
      "email_payment": null,
      "confirmed": true,
      "credential_login": false,
      "language": "en"
    },
    "sso": ["{provider_name}"]
  }
}

We have already seen this type of response, so no need to explain this further. From here on out, we can just query https://sso.playa-games.com/json/client/characters and proceed as before.

With that done, we are now fully able to send any request we want. We only need to do the action in the official client, decipher the request (using CyberChef or otherwise) and look at the output again. For example, fighting another player becomes:

"PlayerArenaFight:{username}/{use_mushrom}

As noted before, this is a time consuming process, but normal game requests have no hidden encoding/hashing/encryption, so it is easy from here on out.

Server responses

Now is a good time to look at what the server actually sends us back, when we login a character, as we would probably like to be able to parse at least some amount of information about a character, before actually sending any further requests. Luckily the server responses are not encrypted, so we can just look at the browser network tab again.

serverversion:2007&preregister:0/0/Europe/Berlin&&ownplayersave.playerSave:492810495/552610/1730242661/1729389836/1547446484/0/0/65537/26/400/200/545237/0/10/0/0/0/12/104/101/5/103/5/3/10/1/0/7/1/8/10/14/18/9/14/0/0/0/0/0/0/0/0/0/0/4/1/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/1/1004/7/11/0/0/0/0/0/0/1/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/1730241348/1/1/1/4/1/5/-1/-142/-45/16/21/2/45/30/45/5/2001/4/0/2/4/1/0/0/0/6/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/85/150/198/36/12/22/0/1730241348/1/1001/4/14/3/4/1/0/0/0/25/0/1/1001/4/14/1/4/2/0/0/0/27/0/4/2001/4/0/5/2/1/0/0/0/26/0/4/2001/7/0/3/1/4/0/0/0/23/0/5/2001/7/0/1/3/4/0/0/0/26/0/4/2001/4/0/4/2/1/0/0/0/23/0/1730241348/9/7/0/0/1/0/0/2/0/0/50/0/12/1/0/0/11/1/0/72/10/0/18/0/8/2/0/0/4/0/0/2/0/0/50/0/17/2/0/0/0/0/0/0/0/0/130/0/12/2/0/0/11/2/0/72/10/0/18/0/8/7/0/0/3/0/0/2/0/0/50/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/7/11/90/0/0/0/0/1729389836/6000/0/0/0/1730243294/0/0/0/0/408/0/0/0/0/0/0/0/0/-111/0/0/4/1730242706/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/6/7/0/0/100/0/0/0/100/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/1729389836/0/0/0/0/0/0/0/0/0/0/0/0/3/0/0/0/0/0/551221/268212/114669/1730243594/0/0/0/0/0/0/0/0/0/0/0/0/0/0/1/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/1730160001/0/10/0/0/0/0/0/9/0/2/0/0/0/0/0/0/0/0/1730205000/1729339200/0/225020000000/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/1/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/&achievement(208):0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/&resources:552610/15/100/0/100/0/0/0/0/0/0/0/0/0/0/0/0&smith:5/0&&messagelist.r:0,admin_news,0,The countdown is on...,1729855800;0,admin_news,1,Black Gem Rush at the witching hour,1730205000;;&&combatloglist.s:27540485,dadad,1,0,1730242694,0;;&friendlist.r:;&login count:8&&sessionid:UVjdF5wU9GNKaBbyRcEMlR24SeNMwyJp&languagecodelist.r:ru,20;fi,8;ar,1;tr,23;nl,16;  ,0;ja,14;it,13;sk,21;fr,9;ko,15;pl,17;cs,2;el,5;da,3;en,6;hr,10;de,4;zh,24;sv,22;hu,11;pt,12;es,7;pt-br,18;ro,19;&maxpetlevel:100&calenderinfo:24/1/8/1/6/1/8/2/8/3/15/1/25/1/19/5/2/1/26/1/28/1/23/1/5/1/6/2/22/1/3/1/3/2/23/2/6/3/21/1&&tavernspecial:0&tavernspecialsub:0&tavernspecialend:-1&dungeonlevel(27):0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0&shadowlevel(21):0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0&attbonus1(3):0/0/0/0&attbonus2(3):0/0/0/0&attbonus3(3):0/0/0/0&attbonus4(3):0/0/0/0&attbonus5(3):0/0/0/0&stoneperhournextlevel:50&woodperhournextlevel:150&fortresswalllevel:5&inboxcapacity:100&&owndescription.s:&ownplayername.r:brukond&maxrank:550616&skipallow:0&skipvideo:1&fortresspricereroll:18&timestamp:1730242706&fortressprice.fortressPrice(13):900/1000/0/0/900/500/35/12/900/200/0/0/900/300/22/0/900/1500/50/17/900/700/7/9/900/500/41/7/900/400/20/14/900/600/61/20/900/2500/40/13/900/400/25/8/900/15000/30/13/0/0/0/0&&unitprice.fortressPrice(3):600/0/15/5/600/0/11/6/300/0/19/3/&upgradeprice.upgradePrice(3):28/270/210/28/720/60/28/360/180/&unitlevel(4):5/25/25/25/&&&petsdefensetype:5&singleportalenemylevel:0&&wagesperhour:10&&dragongoldbonus:30&toilettfull:0&maxupgradelevel:20&cidstring:no_cid&tracking.s:accountlogin&&iadungeontime:6/1725030000/1725822000/1725908400&scrapbook.r:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==&&&owntowerlevel:0&&&&&&&&webshopid:FZpa625C$r482&&dailytasklist:6/1/0/10/1/3/1/10/1/4/0/20/1/3/1/1/2/1/0/1/2/1/0/3/2/17/0/10/2/77/1/5/4/25/0/2/4&eventtasklist:75/1/10/1/57/0/10/1/76/1/10/1/77/1/10/1&dailytaskrewardpreview:0/5/1/24/133/0/10/1/24/133/0/13/1/4/400&eventtaskrewardpreview:0/1/1/26/50/0/2/1/4/800/0/3/2/4/200/28/1&eventtaskinfo:1730073600/1730419199/7&unlockfeature:&dungeonprogresslight(32):-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/0/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/&dungeonprogressshadow(30):-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/-1/&dungeonenemieslight(6):400/15/2/401/15/2/402/15/2/550/18/0/551/18/0/552/18/0/&currentdungeonenemieslight(2):400/15/200/1/0/550/18/200/1/0/&dungeonenemiesshadow(0):&currentdungeonenemiesshadow(0):&portalprogress(3):0/0/0&&expeditions:11/0/0/0/20/16/75/0/11/0/0/0/2/7/150/0&expeditionstate:1/20/1/11/0/0/0/0/3/0/0/0/0/0/0/0/0/20/-5/0/1&expeditioncrossroad:1/1/1/1/11/0&&tutorial_game_entry:3&expeditionevent:1730156306/1730329106/0/1730329106&usersettings:en/0/3/0/a/a&mailinvoice:&&&&cryptoid:0-TXhldzPi6VoXUT&cryptokey:47sF686zf0Z6aOf6

What we see is a mess. A non self describing, custom written encoding, which has missing keys and further custom encodings hidden behind random looking values. Roughly speaking, we have a "&" delimited map of "key:values", where values is a "/" delimited list of strings (that are often numerals). Some values make sense, since the key names are somewhat descriptive, but especially ownplayersave is unnecessarily complicated.

An example

Lets look at the first few values:

492810495/552610/1730242661/1729389836/1547446484/0/0/65537/26

One of these values is the level of the character. Can you guess which one? 26? No. It is 65537. That is because only the lower 16 bits are the level. The upper 16 bits are the amount of arena fights you have done that day...

  • 65537 & 0xFF = 1 // Level
  • 65537 >> 16 = 1 // Arena fights

Just to be clear, I have nothing against bit-packing, bit-flags, or custom compression/encoding. I have actually written my bachelor thesis around these things, but this makes objectively no sense here. The underlying 32bit integer will be send as text, so "/1/1/" would actually be more efficient, than sending "/65537/" and bit shifting the actual values in the client.

The only case, where this is not true, is maybe after reaching level 1000, but that does not happen

Anyways, parsing all of this can be quite time consuming, but possible once you figure out these sorts of tricks and cross reference a few accounts. I am not going into detail for them, since this is just a boring and very time consuming progress.

The only one that I could not solve myself was this:

scrapbook.r:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==

Judging from the name, this is the scrapbook. An item that tracks which items and monsters the character has already encountered. Since this ends with == we are looking at base64 again, but the result is not a valid string. The only thing I could figure out myself was that once you actually start filling the scrapbook, a few values change. More specific, each item and monster corresponds to a bit in this string.

Now, how do you figure out which bit is which item? From what I can tell, there is not correspondence to item ids, or anything else and for this I could not find any help in the decompiled code. Instead, I had to resort to looking at someone else's code. Huge thanks to them and the person they forked this from for figuring this out, because this would have taken me ages to decipher. What you have to do is:

  1. Iterate the characters (bytes) from left to right

  2. Iterate the bits of each characters from right to left

  3. The first 800 bits you look at are monsters, all bits afterwards are items

  4. If the bit is 1, you have seen the item/monster

  5. You figure out which item it is by their bit index:

    1. Roughly every 200 bits, the item category (weapon/shield..) or class (mage/warrior..) will change.
    2. The last 40 of these will be so called epic items, so subdivide the previous range again
    3. The relative position in that range will tell you both the color and model_id of the item. The code looks roughly like this:
    let color = match relative_pos % 10 {
    _ if typ == Talisman || is_epic => 1,
    0 => 5,
    1..=5 => relative_pos % 10,
    _ => relative_pos % 10 - 5,
} as u8;

let model_id = match () {
    () if is_epic => relative_pos + 49,
    () if typ == Talisman => relative_pos,
    () if relative_pos % 5 != 0 => relative_pos / 5 + 1,
    () => relative_pos / 5,
} as u16;

This is awful, error prone and I had no way of testing if it is even correct for all items. What I needed was a tool, that would fill the scrapbook for me and check to make sure the item that it expects to gain is correctly parsed in the scrapbook. As such, I started working on such a tool.

For now though, this post is already long enough and we have covered all the basics needed to decipher requests from the official client, as well as crafting our own requests and parsing the corresponding server responses. Actually implementing all of this amounts to roughly ~16k lines of code in my implementation (sf-api), so if you are planning on making you own API, expect to invest a good amount of time.

Thanks for reading till the end! The next article will be about actually using this API to build a user facing tool/bot and the challenges of building GUI applications with Rust.