Second Life Cache Viewer

A read-only desktop browser for the cache files of the Second Life viewers Firestorm and Black Dragon, written in Python.

It reads the cache directly from disk, so it works while the viewer is running and it never writes to the cache. Everything that is shown comes out of the files the viewer wrote – the texture cache, the object cache, the shader cache, the assets and the name cache.

Contents

• Features
• Requirements
• Installation
• Usage
• The tabs
• How it is done
• Building an executable
• Contributing
• Credits
• License

Features

• Textures – the whole texture cache in a virtualised grid, zoom from the 16×16 thumbnails of the fast cache up to the decoded picture, the exact size of every texture, the state of every entry (complete, truncated, purged …) and a bulk export.
• Objects – the cached prims of every region with their material overrides and the textures they use.
• Shaders – the compiled shaders of the viewer as text, strings and hex dump.
• Assets – every file of the asset cache with its type, the creator of a mesh, the length of a sound and a 3D viewer for the meshes.
• Names – the avatar and group names of the cache, searchable and sortable; the names are also used everywhere a UUID of an account shows up.
• Converted export – sounds become .wav, meshes become .obj with a .mtl, scripts become .lsl, landmarks become .landmark.
• Multi select – every table and the texture grid can be used with ctrl and shift + click, the export buttons write all the selected files at once.

Requirements

• Python 3.8 or newer (tkinter comes with the Windows installer)
• Pillow – used for JPEG 2000 / PNG / JPEG decoding (pip install pillow)
• numpy – used by the 3D mesh viewer (pip install numpy)
• optional: imagecodecs – Pillow can only represent JPEG 2000 images with up to four components. A few Firestorm textures use five components and are only previewed when imagecodecs is installed (pip install imagecodecs numpy).

Installation

Windows

Download the executable from the put it anywhere – it needs no installation. Or clone the repository and use the launcher:

run.bat                                  :: starts the viewer
run.bat "C:\Users\User\AppData\Local\Firestorm_x64\cache"

run.bat looks for a Python 3, installs Pillow when it is missing and then starts cacheviewer.py. install_shortcut.bat creates a desktop shortcut with the icon of the viewer.

Linux

chmod +x run.sh install_desktop_entry.sh
./run.sh                                # starts the viewer
./run.sh ~/FirestormCache               # with a cache folder
./install_desktop_entry.sh              # adds an entry to the application menu

Everything that is Windows specific (the sound output, the file manager, the dark title bar) has a fallback: the sounds are played with ffplay (the volume of the running sound works there as well), aplay/paplay or the default player, and ffmpeg decodes the Ogg Vorbis files.

The settings are stored in ~/.config/FirestormCacheViewer/config.json on Linux and in %APPDATA%\FirestormCacheViewer\config.json on Windows.

Usage

py cacheviewer.py                              :: uses the last used cache folder
py cacheviewer.py <CacheLocation>
py cacheviewer.py --selftest <CacheLocation>    :: console summary, no GUI

The cache folder is the one that contains texturecache, objectcache, shader_cache and cache; it can be changed at any time with the Browse… button and is remembered between sessions.

file	what it does
run.bat	finds Python 3, installs Pillow if needed, then starts the viewer. Any argument is passed on, e.g. run.bat "<CacheLocation>".
run.sh	the same for Linux.
run_silent.vbs	starts the viewer without the black console window; anything it prints goes to run.log.
install_shortcut.bat	creates the desktop shortcut Second Life Cache Viewer (with the icon of the viewer).
install_desktop_entry.sh	adds the viewer to the application menu (Linux), the icon is installed with it.

The tabs

Textures – texturecache/

• texture.entries – 44 byte header (F32 version, U32 address size, 32 byte encoder string, U32 entry count) followed by 28 byte entries (16s UUID, U32 image size, U32 body size, U32 time).
• texture.cache – 600 byte prefix per entry, the rest of the image is stored in texturecache/<first hex digit>/<uuid>.texture. The complete image is prefix + body, exactly like the viewer reads it.
• FastCache.cache – 1040 bytes per texture: S32 width, height, channels and discard level plus the raw 16×16 thumbnail. These are used for the grid, so scrolling stays fast even with 470 000 entries.

The grid is virtualised (only visible tiles are converted to Tk images), the full image is decoded on demand in a background thread.

The Zoom slider (50 % – 400 %, also with ctrl + mouse wheel) changes the size of the tiles. Zoomed out the small 16×16 thumbnails of FastCache.cache are used, when zoomed in the textures that are on screen are decoded in the background and the tiles are replaced with a much better picture as soon as they are ready.

Filter selects Decodable (complete images), Truncated image, Incomplete only (entries that only kept their 600 byte prefix), Body file missing or All usable. Sort can order by cache age, encoded size or UUID. Refresh re-reads texture.entries and the body files.

The grid is navigated with the keyboard: the arrow keys move the selection (and scroll it into view), Home and End jump to the first and the last texture, Page up and Page down move a screen. The tables work the same way, but there Page up and Page down change the page of the table and Home / End jump to the first and the last page.

Several textures can be selected with ctrl + click (shift + click selects a range) and Save PNG / Save raw then write all of them into a folder. The tables use the usual extended selection (ctrl and shift + click, Select all or ctrl + A). The buttons that export several files only appear while more than one row is selected: Export in the Assets tab, Export objects in the Objects tab and Export shaders in the Shaders tab.

The export of the assets converts them instead of copying the raw container: sounds become .wav files, meshes become .obj (with the .mtl), scripts are written as .lsl, landmarks as .landmark and texts as .txt. The original files can still be copied with Copy the original assets unchanged.

The exact image size is read from the JPEG 2000 SIZ marker, which is already inside the 600 byte prefix – so the dimensions are known even when the body file is gone. The dimensions derived from the FastCache thumbnail are only a rough estimate and are shown as such.

A few textures are stored as JPEG 2000 files with five components, which Pillow cannot represent (it reports cannot identify image file). When imagecodecs is installed they are decoded with OpenJPEG instead and the preview shows the first three components.

A texture that is stored with more than three channels (for example the five component textures of the viewer) is shown in the preview window in tabs: the colour image plus every single component (R, G, B, A, …) as a grayscale picture, so that the extra channels can be inspected.

Objects – objectcache/

objects_<x>_<y>.slc holds the cached object data of one region: 16 byte region UUID, S32 entry count and then, per entry, six S32 values (local ID, CRC, hit count, dupe count, CRC change count, body size) followed by the body bytes.

The matching objects_<x>_<y>_extras.slec file contains the GLTF material overrides (GLTFCacheVer:1, region UUID, count and one LLSD/XML document per object). Those overrides are shown in the Extras tab of the selected object.

The Local ID / text box searches either by local ID or for a string inside the cached object bodies. Clicking a column header sorts the table. Show texture opens the texture of the selected object in the Textures tab, Export object writes the body of the selected object and Export cache copies the whole region file (with its extras).

Shaders – shader_cache/

shaderdata.llsd is a binary LLSD document with a shaders map (UUID → binary_format, binary_size, last_used) and a version UUID. The compiled program itself is stored next to it as <uuid>.shaderbin. Both are listed and can be inspected as text, strings or a hex dump.

Names – avatar_name_cache.xml and name.cache

Both files of the cache root are LLSD/XML documents with the names the viewer knows:

• avatar_name_cache.xml – the agents map: UUID → display name, legacy first/last name, user name, the expiry of the display name and whether it is the default one.
• name.cache – the agents map with the legacy first/last names and the groups map with the group names.

The Names tab lists them in a searchable and sortable table, a right click copies the UUID or the name. The mapping is also used to resolve UUIDs everywhere a creator is stored:

• the Creator column of the asset table shows the name of the account that uploaded the mesh (and the UUID as a fallback),
• the info panel of a mesh shows creator: <name> and the UUID,
• the object cache shows the owner of a prim.

Unknown names are looked up when the user asks for it: select the rows in the Names tab and press Fetch from the grid - the creator of a mesh is fetched the same way. The result is remembered in %APPDATA%/FirestormCacheViewer/names.json. The requests are sent one at a time in small batches with a pause between them; a name that the grid cannot resolve is not asked for again for ten minutes, so the service does not see a burst of requests. If the service asks the program to slow down (HTTP 429), the pause before the next request gets longer.

The program does not ask the grid for anything on its own. The Fetch automatically switch of the Names tab (off by default) turns the background lookups on: names that are still missing are then fetched as they show up, for example the creator of a mesh when it is selected in the asset table.

Looking up names on the grid

The public api.secondlife.com/api/agent/... service has been retired, so the viewer asks the simulator instead: the region hands out a GetDisplayNames capability and the viewer queries it with ?ids=<uuid>&ids=<uuid>. The answer is an LLSD document with an agents array and the unresolved ids in bad_ids (indra/llmessage/llavatarnamecache.cpp, LLAppViewer::updateNameLookupUrl).

The Find URL button (next to the url field) looks for that capability in the log of the viewer: the url of a name lookup request is used when the viewer has logged one, otherwise a few of the newest capabilities that are in the log are probed (with a pause between the requests) and the one that answers with an agents map is used. The viewer only writes the request url when the AvNameCache log channel is set to Debug (Advanced → Debug settings), so enable it, let the viewer look up a name and press Find URL again. The log of the viewer is not scanned on its own: only when Fetch automatically is switched on is it checked while the program runs, otherwise Find URL picks up the capability of the current session.

Paste a capability url into the GetDisplayNames URL field and press Fetch from the grid: the selected agents are then looked up exactly the way the viewer does it and the result is added to the list. The url is remembered in the settings, the fetched names in %APPDATA%/FirestormCacheViewer/names.json.

Reload reads the name cache of the viewer again – the app also does that by itself when the viewer writes a new avatar_name_cache.xml, so the names it looks up show up here automatically.

With Fetch automatically switched on, names that are not known are requested as soon as they show up (for example the creator of a mesh in the asset table), the results are cached and every table is updated as soon as they arrive. Without the switch no request is sent to the grid by itself – Fetch from the grid looks up what the table shows.

Assets – cache/

Files named sl_cache_<uuid>_<n>.asset are the assets Firestorm keeps in its generic file system cache. The tab shows the size, the date and the type of every file (the header of all of them is read in the background) and can filter by type. The types are found by looking at the content:

type	how it is recognised
Mesh	12 byte preamble + LLSD header with high_lod/low_lod
Animation	version word 1/2 followed by bone keys (mPelvis, …)
Script	<? LLSD/Binary ?> (LSL script in LLSD/XML)
Landmark	Landmark version 2
Sound	OggS + vorbis identification header
Text	plain text
Texture	PNG / JPEG / JPEG 2000 magic
Json	glTF style JSON

Black Dragon: the cache of the Black Dragon viewer is supported as well. It uses the same texture format (its files are pre-allocated, so texture.entries holds 1 048 576 slots) and the same object cache, but it keeps the asset files in one single folder instead of the folders 0 - f – both layouts are detected automatically.

For sounds the vorbis header is decoded as well (channels, sample rate, bitrate and length). Only the actions that fit the selected file are shown: Play/Stop and the volume slider for sounds, 3D view and Export OBJ for meshes, View image for images, Save file for everything. Play plays the sound inside the application (the Ogg Vorbis file is decoded to WAV and played through the Windows waveOut interface, so the volume can be changed while the sound is running), Export WAV writes the decoded wave file and Save file copies the original asset with the matching extension. The Play button becomes Stop in the same place while a sound is running.

Clicking a column header sorts the table.

The table shows the columns that fit the selected type: sounds get Length, Channels, Sample rate and Bitrate, meshes get LODs, Creator and Ver, animations get the number of Bones. The values are read in the background when the tab is opened (the header of every asset file, about five seconds for 20 000 files), so every column can be sorted by – a second click reverses the order. Rows whose value is not known (a creator whose name was never seen) stay at the end of the table.

The Creator column shows the name of the account that uploaded the mesh (empty when the name is not known yet – the names are taken from the cache of the viewer and are fetched from the grid when possible). The search field finds the assets by their UUID and by the name of their creator (display name, legacy name or user name). While a search is active the required creator names are fetched in the background, a few at a time, and the result is updated as the names arrive.

A right click opens a context menu with the same actions (plus Copy UUID, Copy file path and Open in Explorer). The Textures and the Objects tab have a context menu as well.

Mesh assets (most of them) start with a 12 byte preamble (U32 version, U32 header size, U32 flags) followed by the LLSD mesh header and the mesh data. The header holds the creator, the date and the offset and size of every block:

lowest_lod / low_lod / medium_lod / high_lod   {offset, size}
skin                                           {offset, size}
physics_convex                                 {offset, size, hash}

Every block is a zlib compressed LLSD document. A LOD block is an array of faces, each face a map with

Position        binary, 16 bit positions relative to PositionDomain
PositionDomain  {Min, Max}
NormalizedScale {x, y, z}
Normal          binary, 16 bit normals
TexCoord0       binary, 16 bit texture coordinates
TexCoord0Domain {Min, Max}
TriangleList    binary, 16 bit triangle indices
Weights         binary, skin weights (joint index + weight per vertex)

(the format is the one of indra/llprimitive/llmodel.cpp, the writer is LLModel::writeModel, the reader LLModel::loadModel).

The positions are stored normalised: the uploader shrinks the model into the unit cube (LLModel::normalizeVolumeFaces) and writes the size of the original model as NormalizedScale. Without that scale every mesh would come out as a cube of one metre, so the scale is applied again when the positions are decoded – the exported OBJ and the info panel show the real size of the mesh in metres.

3D view opens a window with a software renderer (z buffer, per face shading and optional texture mapping). Drag with the mouse to rotate and use the wheel to zoom – while the mouse is down a reduced picture of a coarser level of detail is drawn, so the mesh follows the mouse directly; the full quality picture follows when the button is released. A texture can be applied by pasting its UUID – the viewer looks it up in the texture cache and decodes it. Save PNG writes the current view, Export OBJ writes the mesh as a Wavefront OBJ with a MTL file (the applied texture is exported next to it).

Which textures a mesh uses: a mesh asset only contains geometry, it has no texture ids. The textures belong to the prim that uses the mesh: they are in the material of the prim, which Firestorm stores either as a separate material asset or – for the classic object cache – as a material override in the *_extras.slec file of the region. Those overrides are parsed in the Objects tab and list the texture UUIDs per slot (base colour, normal, metallic/roughness, emissive) together with the information whether the texture is in the texture cache.

How it is done

The cache is written by the running viewer, so file counts change while the application is open – press Reload (or Refresh in the Textures tab) to read everything again.

• The viewer also purges textures while it runs: it deletes the body file and frees the entry in texture.entries. A texture that was complete a minute ago can therefore fail to decode later on. The viewer re-reads the entry record every time you select a texture and tells you what happened (body file missing - the viewer purged this texture, purged by the viewer (free slot now), body file truncated, …).
• Entries whose image_size is 0xffffffff are free slots and are hidden. Entries that only kept their 600 byte prefix are marked incomplete and cannot be decoded.
• The viewer records the size of the downloaded file (mImageSize) and the size of the data it wrote (mDataSize). When the two differ, the cached codestream is cut short and no decoder can read it – such entries are listed under Truncated image. The viewer writes those textures again when they are needed.
• All cache files are read with unbuffered handles, because the viewer rewrites them while it is running and a buffered reader could otherwise hand out stale bytes.
• The binary LLSD reader follows the on-disk format: maps are { count k key … }, arrays are [ count … ], integers are big-endian S32, dates are native (little-endian) F64.

The sources

what	where in the viewer source
texture cache	indra/llrender/lltexturecache.cpp
shader cache	indra/llrender/llshadermgr.cpp
object cache	indra/llvocache/llvocache.cpp
material overrides	indra/llvocache/llvocache.cpp (LLGLTFOverrideCacheEntry)
mesh asset format	indra/llprimitive/llmodel.cpp (LLModel::loadModel, writeModel)
material slots	indra/llprimitive/llgltfmaterial.h (GLTF_TEXTURE_INFO_*)
mesh asset preamble	indra/newview/llmeshrepository.cpp
binary LLSD	indra/llcommon/llsdserialize.cpp
names	indra/llmessage/llavatarnamecache.cpp

The files

cacheviewer.py   the tkinter user interface
fscache.py       the format readers (textures, objects, shaders, assets)
meshview.py      the software renderer for the meshes
appicon.py       finds the icon of the application (app_icon.png)
appversion.py    the version of the application
build.py         builds the executable with PyInstaller

Building an executable

build.py creates a self contained executable with PyInstaller:

py -m pip install pyinstaller
py build.py                 :: for the current system
py build.py --windows       :: a Windows .exe
py build.py --linux         :: a Linux binary (on Linux)
py build.py --onefile       :: a single file instead of a folder
py build.py --version 1.2.3 :: use another version number

The result is written to dist/ and is named after the version (FirestormCacheViewer-1.0.0.exe). On Windows the executable also gets a version resource, so the file properties show the same number.

A Windows executable can only be built on Windows and a Linux binary on Linux – or in a container:

docker run --rm -v "$PWD:/app" -w /app python:3.11-slim \
    sh -c "apt-get update && apt-get install -y binutils python3-tk \
           && pip install pyinstaller pillow numpy && python build.py --linux"

Releases

Pushing a tag (v1.2.3) builds the executable for Windows and Linux and puts the files into a The workflow can also be started by hand (Actions → build → Run workflow).

The version number lives in appversion.py; the workflow writes the number of the tag into that file before it builds, so the executable, the title bar and the release always carry the same number.

Contributing

Bug reports and pull requests are welcome. The cache formats are documented in the sections above, so a new format is usually a matter of reading the file and adding a reader in fscache.py.

• keep the viewer read only – it must never write into a cache,
• keep the code Python 3.8 compatible and free of third party modules apart from Pillow and numpy,
• the tables have to stay fast with 500 000 textures, so the heavy work belongs into a thread (see Background in cacheviewer.py).

Credits

• Firestorm Viewer
• Pillow and numpy for the pictures and the renderer.
• PyInstaller for the executable.

This project is not affiliated with Linden Lab, Firestorm or Black Dragon. Second Life is a trademark of Linden Research, Inc.

License

MIT – see LICENSE.

Disclaimer

Please note: under no circumstances will I accept any liability for the usage of this application. This application has been primarily developed using the MiniMax M3.1 Flash, with minor adjustments implemented by myself. Please be advised that the use of this tool and its redistribution is undertaken at your own risk. Should you encounter any issues with the application, please do not hesitate to raise them.