Long Divided, Must Unite: Unified Fake Image Detection Unified FID

Take a guess — which forgery technique was used to produce the chat screenshot below?

Not exactly an easy call, is it? ;)

One thing should be said up front: your answer determines which forgery detection model you would have to call, because the existing research on passive image forensics has essentially been split into several independent tracks, mainly including but not limited to AIGC detection (whole-image generation), (natural) image (partial) manipulation detection, (pure) document (image) tampering detection, and Deepfake (face forgery) detection^[1]. Moreover, models across these tracks are not interchangeable: an image manipulation detection model cannot handle tampered document images, a Deepfake detector does not work on AIGC-generated faces, and so on^[1]. Of course you can force a Deepfake detector onto an AIGC image, but the result is close to random guessing and carries no reference value.

1The Emergence of a Paradox

In practice, then, verifying the authenticity of this chat screenshot runs into a "Preset Paradox":

A user has to decide which forgery method was used in order to call the correct forensic model and obtain a result; but if the user can already predict the forgery method, why would they need a forensic model at all?

The solution to this paradox is also extremely straightforward:

Build a Unified Fake Image Detection (hereafter Unified FID) model, that is, one which makes no assumption about the forgery method and can detect images forged by any means. In practice, a user then only needs to feed the image in question into a single Unified FID detector to obtain the result.

2Division and Reunion in Passive Image Forensics

Before we go deeper into Unified FID, there is one question that must be answered: why did the field of passive image forensics — or content security — develop into (at least) four completely independent tracks? Did researchers simply never consider how these methods are used in practice?

Well — as the old saying goes, the world under heaven, long divided, must unite; long united, must divide.

Just as real-world deployment inevitably pushes every detection solution toward unification, developing fully independent solutions for different forgery methods was also a necessity of technological evolution.

The trajectory of content security / passive forensics / image forgery detection is driven by the shift of forgery-method boundaries from clear to blurred. In short, it is a simulacrum effect brought about by the rapid advance of large models: forgery methods are becoming black-boxed and unified, all delegated to large models.

2.1Division: When the Boundaries of Forgery Methods Were Clear

Rewind the clock just a year or two and you will see that different kinds of forged content basically required different forgery methods. For face forgery, for instance, anything other than Deepfake — Photoshop-based editing, say — was usually fake at a glance and needed no detector at all; the human eye could tell. For document images, one had to Photoshop character by character, keeping fonts, sizes and colors consistent; simply using a generative model would fail to produce even correct glyph shapes.

The requirement at the time was therefore clear: for a given type of image, the forgery method was relatively fixed, and building a dedicated model for each family of methods was not only effective (highly specialized) but also cheap. Gradually, research developed four independent tracks around different forgery methods. There is some overlap (for instance, inpainting in image manipulation already involves direct generation by AI models), but they are largely independent of each other — independent training data, independent model designs, independent benchmark data and protocols — and, crucially, this caused no trouble whatsoever in real-world use.

2.2Reunion: Large Models Have Flattened the Boundaries Between Forgery Methods

Yet with the arrival of unified generation-and-understanding models represented by GPT-Image-2, together with a flood of harness tools, the ability of large models to carry out image forgery directly from user instructions has changed qualitatively. Take data from the insurance industry in Hong Kong, China: today, most high-quality (visually indistinguishable) forged images used for insurance fraud are produced directly by large models — forged medical certificates, forged claim amounts, forged policy records, and so on. This used to be manual Photoshop work, which is expensive to learn and slow to execute and does not fit every scenario; now large models do all of it, and do it remarkably well. As a result, forged images today exhibit the typical profile of "low barrier, large volume, high customization, high deceptiveness" (低大高强).

The capabilities of large models have largely covered — or blurred — the boundaries that once separated image forgery methods: for images of any content, a large model can simply do the job.

3The BP Definition of Unified Fake Image Detection

3.1Is Unified FID Just AIGC Detection?

If everything is now forged by large models, then is Unified FID simply an AIGC detection model?

The answer is no:

The forgery capability of large models comes mainly from harness tooling rather than plain diffusion generation. Leveraging their understanding ability, large models continuously monitor during generation whether the image content perfectly matches the user’s request, and through the harness layer they autonomously call all kinds of tools to fix the image content — matting, repainting, Deepfake face swapping, and so on. Most large models call dozens of tools to refine a single image; Codex, for example, often spends several minutes on one image, precisely because it keeps invoking tools to constrain the generation process. AIGC images, by contrast, are usually defined as whole images generated by a large model in a single pass. That definition carries two conditions: first, whole-image, excluding any secondary or local manipulation that a large model performs on a real image; second, one-shot, ignoring the step-by-step refinements that come from tool calls in a generation–understanding loop.

Second, for a long time to come, what we face on the internet remains a mixed environment: many images are forged by large models, and many are still forged by hand or by small dedicated models. One reason is that a large stock of forgeries from the pre-large-model era already exists. The other is that, despite their growing capability, large models still cannot fully replace manual editing or dedicated tampering tools. For example, in scientific (especially bioinformatics, materials science and related fields) paper image forgery, even faint fluorescence or electrophoresis results at specific image locations carry strong and explicit meaning; large models will struggle for a long while to understand such content precisely and constrain its generation, so in practice people still rely on manual copy-move, splicing and similar tricks (that is, the manual copy-paste and splicing techniques studied under image manipulation detection). We certainly hope a Unified FID model will work on scientific figure forgery — after all, forged samples from scientific papers are scarce, hard to annotate, and extremely harmful. So a model that can only handle large-model forgeries neither satisfies nor realizes Unified FID.

So what exactly is the Unified FID task (the BP definition):

3.2B: Binary Classification

First, in terms of the task paradigm (B: Binary Classification):

Unified FID is an image binary classification task: input an image, output the 0/1 class label for that image.

That is, the paradigm f(x) = y, x = image, y = 0/1.

Within this paradigm:

A standard training set consists of (image, label) / (x, y) pairs, meaning:

1. The model takes in only the image and directly predicts the corresponding 0/1 (real/fake) label.

2. No additional annotations — such as the forgery method — may appear in the training data.

3. Self-supervised / semi-supervised / unsupervised methods that obtain extra information such as the forgery method in order to improve classification accuracy are entirely reasonable and feasible.

4. Using pretrained large models to obtain other relevant information is also reasonable and feasible, as long as the input to both the training set and the test set is the image alone. Using prompt engineering, post-training, or similar techniques to extract extra image information from a foundation model does not affect the task paradigm at all — it is simply a way of strengthening the f(·) function so that it can extract more information from x.

3.3P: Polymath

Second, in terms of model performance (P: Polymath):

Unified FID requires a single generalist model to outperform multiple specialist models simultaneously: one unified model must beat the respective SOTA models on every sub-domain — typically the four we mentioned earlier: Deepfake, AIGC, image manipulation, and document tampering. Only such a model has real application value in practice. Especially in the scenarios particular to China’s very large-scale content platforms — platform governance, fraud-risk control, and public-opinion monitoring — at stages such as interception during moderation, fraud identification, and evidence collection for public-opinion cases, the model faces hundreds of millions of images whose origin and forgery method are entirely unpredictable. Only by surpassing the accuracy of the specialist models can Unified FID be particularly effective in scenarios of this kind.

With the definition in place, Unified FID seems to admit many simple and direct implementations — for instance, picking a base model and running fine-tuning / post-training on a merged dataset appears to suffice.

4A Research Blueprint for Unified Fake Image Detection

So where exactly does Unified FID get difficult? Below we lay out the research blueprint and analyze it point by point:

The technical routes can be divided fairly cleanly into three:

the single compact model route, the large model route, and the compact–large model collaboration route

4.1The single compact model route

This is the route we have studied most deeply, because it rests on a strong domain intuition: all sub-fields of passive image forensics are learning the same thing — artifacts, the traces that forgery methods leave on an image. If what everyone learns is fundamentally shared, and each sub-field already has so many compact specialist models that work well, could we align and merge the datasets and training recipes, pick the strongest compact model, and then train it on a Unified FID dataset to solve the problem?

This idea is highly intuitive, and it is also valuable: the process of selecting the strongest single compact model distills a great many insights, which will be very helpful for building a foundation model dedicated to Unified FID in the future.

But in our tests on ForensicHub, essentially no sub-domain SOTA model meets our P requirement. Even after merging all sub-domain data and training settings, no single sub-domain model is strong enough to dominate the others. Simply put, even after training a sub-domain A model on sub-domain B data, that A model still cannot beat B’s original model. Moreover, across extensive experiments, once we fed data from all four sub-domains (A, B, C, D) into one (A) sub-domain model, its performance dropped noticeably — and the drop on its own (A) sub-domain was especially severe. We observed this phenomenon on ForensicHub, and later named it negative domain transfer in SICA: as soon as training data from other domains is mixed in, model performance necessarily degrades. The existence of negative domain transfer shows that any solution relying on merging existing models and data for joint training is doomed to fail at Unified FID^[2].

But wait — this line of reasoning felt intuitively right, at least at the start. So, was the intuition wrong?

To find out, we ran a large set of experiments in SICA:

First, we linear-probed the feature space of sub-domain models trained on the merged dataset, and found that the feature matrices of all such models show a fairly clear rank problem: after SVD, the top 20 feature dimensions exhibit strong collinearity (too much feature redundancy, insufficient representational capacity), indicating that the models had not learned many useful artifact features.

Second, we inspected the gradients of those models during merged-data training and found clear gradient conflict: data from different sub-domains push the model’s gradient updates in exactly opposite directions, so the gradient descends slowly along a compromise direction — meaning the training dynamics are chaotic and never truly converge or reach an optimum.

Finally, we examined the hand-crafted features that are highly effective and widely used in each sub-domain, and found they are not merely different but mutually non-transferable — for example, the rPPG physiological features used for Deepfake simply cannot transfer to natural images or to document tampering — which indicates that across sub-domains, artifacts are indeed specific in nature.

From these three findings we conclude:

The Ji-Zhe Phenomenon inevitably leads to one consequence in a single compact model — feature collapse: the model ends up learning only the feature representations shared across sub-domains and gives up on the unique (specific) artifact representations of each sub-domain. The artifact features it finally learns are an averaged, degraded representation, so artifact features collapse (to a trivial solution) and are no longer effective across manipulation methods (they lose discriminative power). Simply put, the model can only try to build a common or shared representation for heterogeneous artifacts, and the result is that it works somewhat on every artifact but works really well on none.

4.2The large model route

This is where we invest the most right now, because:

First, the strong out-of-distribution (OOD) generalization of large models is the key to the long-standing generalization difficulty in passive forensics.

Second, large models have an enormous feature space, which can effectively resolve the Ji-Zhe Phenomenon: features can be partitioned into orthogonal or fully disjoint sub-spaces, so the model no longer faces a dilemma between uniformity and specificity, effectively bypassing the feature-space collapse problem.

Besides, large models natively support dialogue, user-intent understanding, and agent invocation, so they hold great potential for iterating on, explaining and auditing detection results and for collaborating with external tools. For example, after making a detection, a large model can automatically produce an explanation of the result, call external physical agents or tools to audit whether that explanation is correct, and then revise the detection result accordingly — forming an agent-based model harness loop, which works beautifully.

Our currently deployed DeFakerOne model follows exactly this idea^[3].

However, training cost aside, the main scientific questions worth pursuing further (see our tech report) lie in two aspects:

Such capability degradation, or model collapse, has extremely serious consequences: the model’s generalization to OOD data, its ability to automatically explain detection results, its ability to revise results according to user context, and its ability to dispatch agents to audit results are all essentially lost. More simply put, the model can no longer speak and can no longer follow user instructions — it can only mechanically perform binary classification on images, completely defeating the original purpose of using a large model.

We are still investigating the underlying mechanism of this model collapse during post-training. This problem should not be unique to forensics; many vertical domains will likely see similar issues emerge as training data accumulates. It is a very valuable and fairly difficult research direction, and discussion is very welcome.

Therefore, how to accelerate inference for Unified FID models under the large model route is also a core problem. Pruning, parallelism, and hardware acceleration — Chinese hardware in particular — are all high-potential research directions under this problem.

4.3The compact–large model collaboration route

This is the most direct, least roundabout route — since every sub-domain already has its SOTA, just let a large model route and dispatch once, aggregate the compact models’ outputs and feed them back to the large model for the final decision (the MoE idea). Beyond our group, many strong teams have taken this route for Unified FID, including (but not limited to) Prof. Jian Zhang at Peking University Shenzhen^[4], Prof. Baoyuan Wu at CUHK-Shenzhen^[5], Prof. Hongtao Xie at USTC^[6], Prof. Jiwei Zhang at BUPT, and others — do follow their work.

This route is currently the fastest-moving and the quickest to yield results. Two scientific questions need further study:

5Closing Words

To summarize, we have outlined the background, definition, three technical routes, the scientific questions behind each route, and potential solutions for the Unified FID task. This serves both as a quick summary of our group’s key research over the past one to two years and as a modest opening statement on the Unified FID problem. A feasibility analysis and potential assessment of the specific technical routes for Unified FID will continue in a follow-up post.

One final point deserves special emphasis: I do not see Unified FID as a replacement for the existing specialist models; rather, the two stand in an upstream–downstream relationship, with Unified FID performing an accurate and rapid first-pass screening and the specialist models then carrying out forensic interpretation — this, I believe, should be the ideal workflow going forward.

Because I have received many emails asking about this topic, my first blog post turned out quite long: it answers almost every question raised in the 30-plus emails I have received so far. There may be omissions and mistakes — please feel free to contact me with corrections.

I will write an English version when I have time; quite a few researchers overseas are also interested in Unified FID. Though I probably won’t get to it any time soon — so I hope they will make do and just run it through a large model for translation. :)

Last but not least,