Recovering Broken Data

Recently, I came across a job description for a software engineer working on tools to recover data from damaged storage devices and corrupted files. The work involved C/C++, understanding binary data structures, analyzing unfamiliar formats, and developing software capable of recovering information that ordinary applications can no longer read. In other words, interesting low-level rabbit holes to explore.

I have never implemented file systems or data recovery software. However, I have recovered data from partially damaged disks by examining raw disk blocks in binary and hex editors. I have also designed a number of binary storage formats for my own applications over the years - from simple record-based object serializations, container formats and various serialized Protobuf object formats, to a speed-optimized binary format for a full-featured word processor. The word processor CrWs, from the early 1990s, beat Microsoft Word and WordStar in speed tests on "large" documents. One of the reasons was its storage and in-memory representation of the documents. It also imported and exported Microsoft Word documents - which was a horrendously complex and ambiguous storage format.

Anyway, when I thought about the task of writing data recovery tools, my thoughts soon drifted into what I would build if this landed in my lap as a general thing.

Definitely not a pool of standalone ancient CLI apps to fix or identify particular problems! Manually inspecting and deducing what's on a disk with 1,000 CLI apps would be slow and a waste of time.

The problem with specialized tools

There are lots of tools for analyzing and recovering data. Some understand file systems, others can recognize file signatures (like file on Linux), and some specialize in particular areas.

These tools are useful. Significant time and expertise have been invested in some of them. But each tool usually understands only a tiny part of the problem.

Why not design something that solves the actual problem?

The thing I would build is a general app that can work on disk images, individual files and arbitrary binary blobs and provide some useful insight.

I would start with existing libraries to read and validate file systems and various file formats. Then I would use an AI agent to check and patch the error handling in each of them, so the tool would not just bail out as soon as an error was detected - but also attempt recovery, if possible, by scanning for more valid data when a block of garbage or disk read errors prevents the app from seeing the original bits. For some formats, this would be relatively simple; for others, pretty hard or impossible - but even without recovery, the tools could be able to detect what the bytes are. And that in itself is a goal.

Then wrap each tool into a standalone binary so it can run as a constrained process (to prevent, as far as possible, any zero-day exploits in libraries - and destructive bugs - from escaping into my desktop environment) using a uniform API. That way my app would be able to probe a disk image or binary blob relatively safely with hundreds of tools - where many could run simultaneously in total isolation - and report the findings back to my app.

For example, parsers for NTFS, ext4, FAT, ZIP, SQLite, MySQL binary databases, PostgreSQL and definitely Microsoft's old OLE Compound File format. (Those old Windows document formats deserve some special attention. Anyone who has studied the internals of legacy binary formats knows that they can be quite entertaining.)

A parser might identify a directory structure at a particular offset. Another might recognize a document header inside a damaged allocation block. A third might identify a sequence of records that could belong to the same file.

Individually, these discoveries may not tell us much. Combined, however, they could tell us quite a lot.

A common model for what we find

Every parser would produce structured information using a common interface.

For example:

• The location and size of a recognized structure.
• The format and version it appears to belong to.
• References to other structures or data ranges.
• Checksums and validation results.
• Missing or corrupted fields.
• Alternative interpretations of the same data.
• Information about how the conclusion was reached.

I would also distinguish between verified facts and hypotheses.

For example, finding a known magic number at a particular offset is a fact. Concluding that the following bytes represent a complete document is a hypothesis that needs more evidence.

The analysis engine could maintain relationships between these findings. Some structures refer to other structures. Some interpretations contradict each other. Some discoveries support a hypothesis about how the original data was organized.

Basically, I would make a graph of what we know, what we think we know, and how the different pieces of information are related.

This would allow us to investigate data without committing too early to one interpretation.

Visualizing the data

I have worked a lot with C++ and Qt, so that would be my natural choice.

I would probably design a number of views into the data. One of them would be a high-level view of the entire disk image or binary blob, with colors and symbols representing different kinds of information.

Perhaps blue for recognized metadata, green for validated data structures, yellow for fragments that appear recoverable, red for corrupted areas and gray for regions we haven't identified yet.

I could then zoom into a region, inspect its contents as hexadecimal data, and see what the different parsers have discovered.

If two parsers disagree about a particular region, I would want to see both interpretations.

Clicking on a pointer or reference should take me to the target structure. If the reference appears invalid, I would want to search for plausible targets elsewhere in the data.

And if I find an interesting pattern, I should be able to select a region and ask the system to look for similar patterns throughout the image.

I think visualization is crucial here. People are good at recognizing patterns. But looking at gigabytes of hexadecimal data does not trigger that ability for most of us. I only knew one individual who could intuitively read hexdumps and convert that to assembly code or non-textual information in his brain. I need visualizations, and in some cases clear text, to quickly recognize patterns.

And then there is AI

AI is really awesome and quick at picking up patterns it recognizes from its training, reverse engineering binary data (compiled binary programs) into source code in your favorite programming language, and probably also other well-known binary formats. So making a recovery tool today without utilizing AI in the tool itself (not just to write parts of its code) would be madness. However, it's not magic. It will not break proper encryption - and pushing it hard on random binary data could trigger some really amazing hallucinations.

I would bake an agent with a really good harness right into the core of the tool, and also give it a fat MCP server. There is no standard C++ library for MCP servers - I wrote my own MCP server library for my own projects. Today I even use this for integration tests of complex C++ projects, using Python (look! No AI!) for the tests.

The AI agent in such an app must be capable of running long sessions using multiple AI models, and have its own orchestration and delegation. It should be able to explore multiple assumptions and questions in parallel, and change source and focus based on their outcomes.

MCP is a standardized interface for AI agents to use "tools". I would expose all the tools in the app's toolbox to the MCP interface - so the built-in AI agent - and even external agents - could probe the data and try to make sense of the data returned. The nice thing with MCP is that most new models today know how to use them and their tools - even new tools that did not exist when the model was trained.

There already exist AI models trained directly on binary data, like FiFTy and ByteSCAN. These can recognize file types from arbitrary fragments, even without intact file headers. That opens up an entire avenue of new opportunities for deducing what some binary blob actually is. If I built a sustainable business around this, I may even consider creating my own small models to make recovery cheaper and faster.

The app could use a deep reasoning model to handle ambiguities when several probes give conflicting conclusions about a file or a blob. It could pursue each alternative in depth, trying to recover parts of the data and evaluate if the resulting data makes any sense. If not, it may ask a human for help ;) In many cases it may end up converting data from unknown sources that looks like text into plain-text files (text, Unicode, UTF-8, JSON, HTML, etc.), removing binary noise surrounding the text blocks. A customer may want that - and use its own domain knowledge to turn those text files back into structured data.

One really powerful feature could be to find the source code for whatever app wrote a blob that the app does not yet understand - if the app is open source, or the company has access to the code. From the code, it should be possible for the AI to directly test assumptions about the binary data, and to write a parser/scanner/converter to check and extract whatever data is still readable.

Using all these tools together, the system could first determine the partitioning and file systems on each partition on a disk. Then recover the database file(s) for some storage engine, and then extract, for example, individual old Microsoft Word documents from partially recovered database rows, recover whatever information still remains in the individual documents and convert them to a modern document format - complete with markings of the missing segments.

With AI, it may also be possible for the app to create a compiled (=fast) program, for example in Rust, to do the actual data recovery when all the analysis is complete, with or without oversight and approval of a human operator.

Security and reliability

There is one rather obvious problem with this design. We would be feeding corrupted, potentially malicious data into a collection of complex parsers - some of which might be modified to continue processing input that they would normally reject. That is a wonderful way to discover and experience security vulnerabilities.

I would run parsers in isolated processes, with limited resources and access to only the data they need. The original disk image would remain read-only, and any reconstructed data would be written separately.

The AI agent would also need restrictions. There are many ways to trick an AI into doing bad things. So for this job, most of the AI-assisted work could be done without exposing any mutable tools to the AI (i.e., no access to call tools that can change anything, including shell access).

For sensitive data, it makes sense to use local models - you don't want to send medical records or business secrets from your best customer to Google, Microsoft, Anthropic or OpenAI. A local model in this context doesn't mean one that runs on your laptop or workstation - but one that you control. It can still run in a data center, even though I personally would prefer one running in my office.

Another thing I would add is traceability. If the system reconstructs a record, I want to know exactly which bytes were used, which assumptions were made, and what tests passed or failed.

Ideally, every result should be reproducible without involving the AI at all.

For better security, all the computers working on a recovery job could be on their own, local network - without any physical connections (including Wi-Fi and Bluetooth) to the Internet or any other networks. That way you know with 100% certainty that they won't escape your lab, create a swarm and hack the world. Or even leak sensitive data - at least not until some data leaves the lab on a USB disk or DVD containing a clever virus. (If an AI really wants to break free, it will be almost impossible to stop it - I won't even speculate about how clever it could be).

Solving the actual problem

This idea started with me looking at a feed of remote job openings, circling in on one that looked like a fun challenge, and then letting my thoughts wander while ChatGPT spent a few minutes tracing the original posting and evaluating the requirements against what it knows about me, and the job's potential fun factor against my preferences.

That's how my mind works. I get a flicker of input that triggers an idea, and a few minutes later I have the outline of a new project or product. That's why, by the way, I wrote smtool Voice; to capture these ideas while I still have them fresh in memory.

My initial reflex was to zoom out from the idea of making tools to solve a small bit of the overall problem, to a tool that addresses the bigger picture. That tool would, in a real implementation, contain hundreds or thousands of specialized parsers and analyzers that could be called upon. So it's not a small or easy thing - although there is probably lots of code that can be used unmodified or with small patches to address the various file systems and application storage formats.

I have no idea about what products and projects exist around data recovery. I'm pretty sure smart people have spent considerable time making tools that are awesome and thorough.

My idea may be total crap, or it may be a 100 million euro business. After all, this whole idea spread through my neurons in a few minutes while I waited for ChatGPT to research something. I have learned a few tough lessons over the years: not to start an actual project before I do some research and ask myself tough questions. Or even discuss the idea with smart people.