2026-10-16 Decrypting FortiOS 8.0.x Firmware Image
The materials here were partly given out in my talk “Cracking Hard(ware) : The World of Network Appliance Forensics” at STANDCON 2026 in Singapore, this is a partial writeup for those who missed it.

The materials here were partly given out in my talk “Cracking Hard(ware) : The World of Network Appliance Forensics” at STANDCON 2026 in Singapore. This is a partial (but more focused and indepth, i think, on the important stuff) writeup for those who missed it. I will be mainly focusing on my work in the Fortigate FortiOS appliance as its one of the few parts of the presentation that alot of audience members had found to be interesting and novel enough to current industry applications.
Preface
2025 was a break out year for attacks on network appliances, with network appliance vulnerabilities increasing nearly eight fold according to Mandiant. As a matter of fact, Singapore last year had made the rare move of calling out a specific advanced persistent threat (APT) group, UNC3886, as a recent perpetrator of an attack in Singapore’s critical infrastructure. This is the first time Singapore has named a threat actor targeting its critical infrastructure in a public statement.
Despite this, doing forensics work on these appliances are a nightmare. To comply with standards such as the Federal Information Processing Standard Publication (FIPS), vendors are required to encrypt the full disk of appliance. Some appliances even go further, with custom encryption schemes and some interesting trickery to prevent people from tampering their appliances. Terms of Service of these devices also always prohibit tampering, even to do your own investigations. This is all the while vendors give little to no support outside of patches, sometimes they will offer to take your device back and replace it for you or analyze your logs, but these take time especially in a period of mass exploitation and may not give you a full picture of who and what actually hit your organization.
In my talk i’ve shown the insane lengths I’ve taken to actually acquire these appliances, from scrounging the depths of Indonesian carousell to physically deadlifting a 1U appliance through customs as a carry on (funfact, airport security 100% allows you to carry enterprise servers as a carry on, they’ll just look at you abit weird).

If you worked in datacenters, offices big and small, and many other institutions I’m sure you’ve come to know Fortigate. There have been many cases where we’ve seen appliances being backdoored to spy on email servers, to siphon credentials, but if you really think about it the most elegant use of a backdoored appliance is as a remote access vector to a secure network.
Within the industry, Fortigate has a has been the butt of the joke of the infosec community due to how common zero day CVEs seem to impact Fortinet appliances and services. This is of course due to the ubiquitous nature of the appliance aswell as their role in the network, which have made them become a very highly prized target for major global APT groups. Which is why forensics and analysis towards the firmware of Fortigate’s primary firmware (FortiOS) could be beneficial for the modern DFIR guy to learn.
When talking about Fortigate, they are way more restrictive than other vendors their appliance and the firmware that goes alongside it. And even though its a pretty common appliance, currently non-EOL models still go on marketplaces locally for obsecene amounts of money. I wasn’t about to give up a few months of my rent money just for a curiosity. And also for firmware upgrades, they are locked behind an active support contract, which is Forticare. So we are gonna need a better solution for this.
I was told by a friend that Fortinet actually offers a virtual version of its appliance on AWS as an Amazon Machine Image, which you can even try for free. This is intended for those who want to use Fortigate firewalls in lieu of AWS’s first party networking security tools, which is a use case that I wasn’t even aware of before this case.

The firmware
When we tree a FortiOS directory, we see a few things inside. The presence of .chk files alongside the main components indicates signature verification mechanisms, while the .db and .db.x files contain integrity databases and certificate information respectively. Unlike other network appliances in the market, FortiGate appliances uses a weird non-standard encryption scheme, RSA signature verification, and multiple integrity checkpoints that activate during different phases of the boot process.1 So to extract the data, we need to extract and examine the flatkc file, which contains the Linux kernel modified by Fortinet. Unlike standard Linux kernels that just uses LUKS, FortiGate’s implementation includes custom functions for ramdisk verification and decryption.
~/fortios$ tree -a -L 1 ~/fortios/fs
/home/user/fortios/fs
├── bin
├── boot
├── boot.msg
├── cmdb
├── config
├── datafs.tar.gz
├── datafs.tar.gz.chk.bak
├── .db
├── .db.x
├── dhcp6s_db.bak
├── dhcpddb.bak
├── dhcp_ipmac.dat.bak
├── etc
├── extlinux.conf
├── filechecksum
├── flatkc
├── flatkc.chk
├── ldlinux.c32
├── ldlinux.sys
├── lib
├── log
├── lost+found
├── rootfs.gz
├── rootfs.gz.chk
└── web-ui
Bishop Fox and Optistream had done early works in decoding the FortiOS firmware, and in April 2024 GreyNoise Labs documented a way to decrypt FortiOS 7.0.x.2 There the firmware used stock ChaCha20, and the whole cipher state (a 32-byte key and 16-byte IV) sat in the clear in the kernel (flatkc). You could recover it with lief to resolve a symbol offset and objdump to dump the static memory. But the newer FortiOS 8.0.x firmware seems to introduce stronger protections against firmware decryption, as searching for the usual ChaCha20 fingerprint (the constant "expand 32-byte k" (0x61707865 0x3320646e …)) turns up nothing. The secure-boot and integrity path of the OS has seemingly been rebuilt around a root of trust
- The symmetric key is now RSA-wrapped and appended to
rootfs.gz, and the public key used to unwrap it is itself obfuscated in the kernel - ChaCha20 and AES were dropped for a customized RC4 variant
The verifier sub_FFFFFFFF81710483() delivers the 32-byte disk key as a 256-byte block appended to the end of rootfs.gz. Since the image can only be verified with the public exponent, that block was produced with Fortinet’s private key. This is RSA signing with message recovery, not encryption. The decryption core sub_FFFFFFFF81710334() initializes a standard 256-byte S-box but passes the output through an obfuscated keystream function. The customization is entirely in the output stage.
However, on 8.0.x the symbols are gone. The new code seems to be suggest that if the firmware verification fails, the device must brick. So the verifier sits one or two hops up the call graph from a halt. We can search the kernel image in IDA for machine_halt() / panic() / emergency_restart(), then cross-reference it. One short walk lands on sub_FFFFFFFF81710483(), a function is full of unnamed variables, but there seems to be a 270-iteration for loop carrying an & 0x1F (mod-32) XOR.
rsa_parse_pub_key(v36, n6291648_1, 270);
The XOR rebuilds the RSA public key that the next call goes on to parse. Double-clicking the two source pointers gives you both inputs, a 32-byte de-obfuscation seed at byte_FFFFFFFF8179A2C0 in .init.data (e.g. 5C 19 C6 E1 …), and the 270-byte obfuscated certificate right next to it at byte_FFFFFFFF8179A1A0
270 is not just a random byte size, it is the exact length of a DER-encoded PKCS#1 RSAPublicKey (the bare SEQUENCE { modulus, publicExponent }, not a SubjectPublicKeyInfo wrapper) for a 2048-bit modulus with a 3-byte exponent.
So the 270-byte blob is a raw DER public key, XOR-masked byte-for-byte, and it must begin 30 82 01 0A once unmasked. That gives a free correctness check before touching RSA at all. The & 0x1F loop is a repeating 32-byte XOR keyed by the seed.
uint8_t pubkey_der[270];
for (int i = 0; i < 270; i++)
pubkey_der[i] = obf_cert[i] ^ seed[i & 0x1F]; /* seed = 32 B @ 8179A2C0 */
rsa_parse_pub_key(&rsa_ctx, pubkey_der, 270);
So in a script we can do something like :
def deobfuscate_pubkey(obf_cert: bytes, seed: bytes) -> bytes:
der = bytes(obf_cert[i] ^ seed[i & 0x1F] for i in range(len(obf_cert)))
assert der[:4] == bytes.fromhex("3082010A"), "seed or offset is wrong"
return der # load with RSA.import_key(...) as a DER RSAPublicKey
The seed at …8179A2C0 is 32 bytes and lives in kernel data, exactly like the disk key, so a good guess would be that it is the key for the big blob. It isn’t, as it only unmasks the public key in memory. The DER-header assertion proves this, a wrong 32-byte value won’t XOR the certificate into a valid SEQUENCE. The second trap is an off-by-one in the payload slice. After grabbing the 256-byte tail and running m = c^e mod n, you recover a structured block, and the C reads the key at a fixed front offset, v11 + 223.
An RSA-2048 recovery returns a PKCS#1 block that always starts 00 01, so byte 0 is always 0x00. That makes the value numerically a 255-byte integer (m < 2^2040), and the m.to_bytes(256, "big") pads the leading 00 back and the key sits at offset 224, while the device’s buffer keeps the 255-byte form with no leading 00, where the key sits at offset 223, exactly that hardcoded v11 + 223. Reuse the 255-byte offset against a 256-byte buffer and every field is one byte early, so the key comes out shifted and the output becomes uniform garbage. The 00 01 header is guaranteed, so the mismatch is always exactly one byte and hits every image. The fix is the standard message-recovery rule, anchor to the tail as recovered[-32:] is the last 32 bytes regardless of the leading zero, so it coincides with v11 + 223 without having to track the length.
Put together, the whole verifier decompiles to roughly this — the unnamed locals are IDA’s, the loop and the two machine_halt() edges are the fail-closed skeleton:
__int64 __fastcall sub_FFFFFFFF81710483(_BYTE *rootfs, unsigned __int64 len)
{
int i; // [rsp+1Ch]
__int64 v11; // [rsp+28h] base of the recovered struct
char pubkey_der[270]; // [rsp+40h]
_BYTE recovered[256]; // [rsp+160h]
_BYTE digest[32]; // [rsp+260h]
for ( i = 0; i < 270; ++i ) // de-obfuscate the embedded DER public key
pubkey_der[i] = byte_FFFFFFFF8179A1A0[i] ^ byte_FFFFFFFF8179A2C0[i & 0x1F];
if ( rsa_parse_pub_key(&rsa_ctx, pubkey_der, 270) < 0 )
machine_halt(); // not a valid SEQUENCE -> brick
v11 = sub_FFFFFFFF817103F0(&rsa_ctx, &rootfs[len - 256], recovered); // m = c^e mod n
sha256(rootfs, len - 256, digest);
if ( memcmp(digest, (_BYTE *)(v11 + 160), 32) ) // != recovered[160:192] -> brick
machine_halt();
sub_FFFFFFFF81710334(rootfs, len - 256, (_BYTE *)v11 + 223); // key = recovered[224:256]
return 0LL;
}
With the slice corrected, the trailing 96 bytes resolve into three aligned fields:
recovered = pow(c_tail, e, n).to_bytes(256, "big")
# [00 01] [FF … FF] [00] || <---------- payload (96) ---------->
# SHA256(body) | field_B (32) | RC4 key (32)
# [160:192] [192:224] [224:256]
expected_sha256 = recovered[-96:-64] # 160:192
rc4_key = recovered[-32:] # 224:256
The middle 32 bytes ([192:224]) are recovered but unused by the decryptor. Given the layout, the candidates are a second digest (SHA256 of the decrypted body, as a post-decrypt check), a per-image nonce, or a build/model tag binding the key to one firmware. Diffing that field across two images settles it: constant means a tag or reserved field, per-image means a digest or nonce. The loop itself is self-verifying, you only trust the extracted key if SHA256(encrypted_body) equals recovered[160:192], which confirms the offsets. The key schedule uses key[i & 0x1F], which is ordinary RC4 key scheduling with the key length hardcoded to 32 (% keylen collapses to & 0x1F). In the x86_64 disassembly of sub_FFFFFFFF81710334() it shows up as an and eax, 0x1F inside a 256-iteration loop that also swaps S-box entries. That pairing is the signature for “RC4-family, 32-byte key,” and it’s the first thing to grep for.
The custom part seems to be in the PRGA output, and the classic RC4 byte is still in there. After the swap, S[i] = old_sj and S[j] = old_si, so the standard RC4 output S[(S[i] + S[j]) & 0xFF] is really just S[(old_sj + old_si) & 0xFF] which is comp_z in the code. So Fortinet does the normal RC4 thing first, computes the plain RC4 byte, and then whitens it by running it through two extra table lookups:
The addresses 0xFFFFFFFF81838080 and 0xFFFFFFFF81838078 correspond to __initramfs_start and __initramfs_end respectively, indicating where the ramdisk is loaded in memory. Cross-referencing these variables reveals two critical functions: fgt_verify_initrd and fgt_verify_decrypt, which handle signature verification and decryption respectively. Looking at it closer, the core looks suspiciously like RC4 in its setup and swaps.
void __fastcall sub_FFFFFFFF81710334(_BYTE *buf, __int64 len, _BYTE *key)
{
int i, j;
__int64 k;
unsigned __int8 S[256];
unsigned __int8 si, sj, v8, v9, x, y, z;
for ( i = 0; i < 256; ++i )
S[i] = i;
for ( i = 0, j = 0; i < 256; ++i ) // KSA, key length fixed to 32
{
j = (unsigned __int8)(j + S[i] + key[i & 0x1F]);
si = S[i]; S[i] = S[j]; S[j] = si;
}
i = 0; j = 0;
for ( k = 0; k < len; ++k )
{
i = (unsigned __int8)(i + 1);
si = S[i];
j = (unsigned __int8)(j + si);
sj = S[j];
S[i] = sj; S[j] = si;
z = S[(unsigned __int8)(sj + si)];
x = S[(unsigned __int8)(j + sj)];
v8 = S[(unsigned __int8)((j >> 3) ^ (32 * i))];
v9 = S[(unsigned __int8)((i >> 3) ^ (32 * j))];
y = S[(unsigned __int8)(v8 + v9) ^ 0xAA];
buf[k] ^= x ^ (unsigned __int8)(y + z);
}
}
The disassembly tells for the added layer are shr …, 3 / shl …, 5 (the >>3 / <<5 on the counters) and a literal xor …, 0xAA, none of which appear in stock RC4.
; sub_FFFFFFFF81710334
shr eax, 3 ; j >> 3 ┐ idx1 = (j>>3) ^ (i<<5)
shl edx, 5 ; i << 5 ┘
xor eax, edx
movzx ecx, byte [S+eax] ; S[idx1]
; … idx2 = (i>>3) ^ (j<<5) ; S[idx2] the same way …
add ecx, esi ; S[idx1] + S[idx2]
xor ecx, 0AAh ; ← whitening tell, absent from stock RC4
movzx ecx, byte [S+ecx] ; comp_y = S[(S[idx1] + S[idx2]) ^ 0xAA]
The aim seems to be to keep a proven cipher core, add whitening so the emitted keystream matches no reference RC4, and defeat constant- and signature-matching without designing a cipher from scratch. Once the key boundary is aligned and the PRGA operator is reproduced, the SHA256 computed over the firmware body matches the digest recovered from the RSA block.
Extraction Script
import hashlib
from Crypto.PublicKey import RSA
CERT_FILE = "forti_pubkey.der" # de-obfuscated DER RSAPublicKey (270 B)
ROOTFS_ENCRYPTED_FILE = "rootfs.gz" # original encrypted firmware
ROOTFS_DECRYPTED_FILE = "rootfs_decrypted.gz"
# 256-byte RSA-wrapped block from the tail of rootfs.gz
rootfs_tail_hex = (
"31C6C1E3E33F19F477E38CB15845DFC919C60D0CE33975E34A63815BB8BCB47C"
"4FDCA8AC8F30C64053C00A045FA876F2B3C2D0AA2D82B64842E2C6B43C7A4A6C"
"169577A860171DE17019EE1CE7C89E6F44833B959104E4E0B61525C214EA68AE"
"D4557BC902597E15B8C5A38C18217FB0E9F77EE1F3E50CEE4D57166DC6FA358B"
"5E6FAF48B11D26399209D792018729E1AF9040C14026250EF18DA11EBE14A8F9"
"B9D2F002B2772DA82796E755DCFD238DD563611C323175ED4E238B23EB76D6FA"
"EE0A6FB93EA9CC5149A62AB2685B44ECFB43BE3185D3976B0CE58E6BC9A2B01F"
"0D8D2BCA4CD928C9F874E4F6D5A81460257C31C6CE2CC09B49F772878E08BEA0"
)
def deobfuscate_pubkey(obf_cert: bytes, seed: bytes) -> bytes:
"""Rebuild the DER RSAPublicKey from the 270-B obfuscated cert (byte_..A1A0)
and the 32-B seed (byte_..A2C0). Must start with SEQUENCE 30 82 01 0A."""
der = bytes(obf_cert[i] ^ seed[i & 0x1F] for i in range(len(obf_cert)))
assert der[:4] == bytes.fromhex("3082010A"), "seed or offset is wrong"
return der
def forti_custom_rc4_decrypt(ciphertext: bytes, key: bytes) -> bytes:
# --- KSA: standard RC4 schedule, key length hardcoded to 32 (i & 0x1F) ---
S = list(range(256))
j = 0
for i in range(256):
j = (j + S[i] + key[i & 0x1F]) & 0xFF
S[i], S[j] = S[j], S[i]
# --- PRGA: classic RC4 byte (comp_z) whitened by two extra lookups ---
plaintext = bytearray(len(ciphertext))
i = j = 0
for idx, c in enumerate(ciphertext):
i = (i + 1) & 0xFF
old_si = S[i]
j = (j + old_si) & 0xFF
old_sj = S[j]
S[i], S[j] = old_sj, old_si
comp_x = S[(j + old_sj) & 0xFF]
idx1 = ((j >> 3) ^ (i << 5)) & 0xFF
idx2 = ((i >> 3) ^ (j << 5)) & 0xFF
sum_idx = (S[idx1] + S[idx2]) & 0xFF
comp_y = S[(sum_idx ^ 0xAA) & 0xFF]
comp_z = S[(old_sj + old_si) & 0xFF] # == standard RC4 output
keystream = comp_x ^ ((comp_y + comp_z) & 0xFF)
plaintext[idx] = c ^ keystream
return bytes(plaintext)
# --- Load the reconstructed public key -------------------------------------
print("Parsing the RSA public key...")
with open(CERT_FILE, "rb") as f:
pub = RSA.import_key(f.read())
n, e = pub.n, pub.e
print(f" N (first 16 hex): {hex(n)[:18]}...")
# --- RSA message recovery over the tail block ------------------------------
print("Recovering the wrapped payload (m = c^e mod n)...")
c_tail = int.from_bytes(bytes.fromhex(rootfs_tail_hex), "big")
recovered = pow(c_tail, e, n).to_bytes(256, "big")
# Trailing 96 bytes = [ SHA256(body) | field_B | RC4 key ], tail-anchored
expected_sha256 = recovered[-96:-64]
rc4_key = recovered[-32:]
print(f" expected SHA256 : {expected_sha256.hex()}")
print(f" core RC4 key : {rc4_key.hex()}")
# --- Self-verifying integrity check ----------------------------------------
print("Reading and verifying firmware body...")
with open(ROOTFS_ENCRYPTED_FILE, "rb") as f:
full = f.read()
encrypted_body = full[:-256]
actual_sha256 = hashlib.sha256(encrypted_body).digest()
print(f" actual SHA256 : {actual_sha256.hex()}")
if actual_sha256 == expected_sha256:
print(" SHA256 match — offsets and key are correct.")
else:
print(" SHA256 MISMATCH — check tail slice / payload offsets.")
# --- Decrypt ---------------------------------------------------------------
print("Decrypting the filesystem with the RC4 variant...")
decrypted = forti_custom_rc4_decrypt(encrypted_body, rc4_key)
with open(ROOTFS_DECRYPTED_FILE, "wb") as f:
f.write(decrypted)
print(f"Done — written to {ROOTFS_DECRYPTED_FILE}")
Why does Fortinet do This?
In the posts by Bishop Fox and GreyNoise, both cannot figure out why Fortinet designed their firmware encryption scheme like this. The truth is that a FortiGate appliances need to be able to boot in airgapped environments, so sometimes it doesn’t have the luxury to phone home for a decryption key. The appliance has to carry everything needed to decrypt its own rootfs.gz, which means you can never get real confidentiality of the filesystem against someone with the firmware image. In the 7.0.x firmware version, the simple encryption scheme was probably meant to keep casual binwalk readers out. Which is probably why the key was simply hardcoded.
The 8.0.x seems to try to wrap the key instead of leaving it as a plaintext (so only Fortinet’s private key could have produced it), the public key is XOR-obfuscated so the anchor is harder to find, and the whitened RC4 was used so there’s no magic constant or reference keystream to match. Crucially it’s RSA signing with message recovery, not encryption because the goal was to prove the image came from Fortinet. In this case i think they did well enough, but this demonstrates that firmware compromise of perimeter devices such as network appliances are becoming more common and defenders need to actually start preparing not only to defend their networks from these attacks but learn how to properly respond and investigate these types of incidents.
So what did we learn from all this? Probably that all of this is just very hard. Cryptographic function cracking, firmware decryption, linux kernel analysis, if you’re able to do all of that you won’t settle working in incident response. You would be working in vulnerability research to find zero days and easily make ten times more. The unfortunate truth is that so far there hasn’t really been an effective way to detect, analyze, and tackle these types of attacks.
NIST SP 800-193, Platform Firmware Resiliency Guidelines (May 2018), which specifies the protection, detection, and recovery properties a device’s firmware is expected to enforce; see also NIST SP 800-147 (BIOS Protection Guidelines), NIST SP 800-53 Rev. 5 controls SI-7 (software, firmware, and information integrity) and SC-13 (cryptographic protection), and FIPS 140-3 for validated cryptographic modules. ↩︎
GreyNoise Labs, Decrypting FortiOS 7.0.x (23 Apr 2024), https://www.labs.greynoise.io/grimoire/2024-04-23-decrypting-fortios/, building on earlier reverse-engineering of the FortiGate firmware container and update packaging by Bishop Fox and Optistream. ↩︎