Explainable AI in the software chain: detecting anomalies is not proving security

The software chain grows faster than the human ability to inspect each version of each dependency. A detector can help you prioritize what to investigate. The risk begins when your score becomes a verdict and your plausible explanation becomes proof that the packet is malicious — or safe.

Status: Evidence Brief approved for publication. This text analyzes an article by *Scientific Reports*. It does not claim that Trustyu Forge has reproduced the results, implemented the method, or validated its effectiveness in production environments.

What XAI-SCS proposes

The paper presents a hybrid architecture with CNN, LSTM and autoencoder to analyze signals from npm and PyPI package versions. The explainable AI layer seeks to show which characteristics contributed to an alert, allowing a practitioner to investigate the hypothesis rather than just receiving an opaque score.

The dataset brings together 12,000 package versions. The labels derive mainly from CVEs and simulated poisoning with GAN. In separate internal testing, the authors report F1 of 0.91; in five runs, the average was 0.90 with a deviation of 0.02. The average latency reported was 0.8 seconds on gated streams representative of CI/CD.

These numbers describe the experiment. They don't tell you how many real incidents would be prevented, how many legitimate packages would be blocked in an organization, or how long it would take a team to acknowledge or dismiss alerts.

Generalization is still the critical question

The authors also assembled an external set of 240 real software chain incidents without CVE, manually curated. This step is relevant because known CVEs and synthetic samples can teach the model to recognize history, not the next attack. Still, the external set is small, specific and described as a preliminary assessment.

In security, generalization needs to survive changes in ecosystem, language, maintainer, publication pattern and adversarial behavior. A high F1 in a controlled cut is not certification for another company's release stream.

Explainability helps — and can also mislead

One explanation might make the alert auditable: unusual maintainer change, abrupt dependency growth, metadata divergence, or strange pattern in the content. This helps the analyst formulate the next question and record why they decided to block or release.

But explanation methods typically describe the contribution of features to the model's output. They do not prove causality, malicious intent, or completeness. A coherent narrative can still be wrong. The explanation should point to verifiable external evidence — package, signature, diff, provenance, isolated execution, and repository context — and not end the investigation.

What the developer survey measures

The study consulted 82 developers. Clarity received an average of 4.3 out of 5 and actionability, 4.1. Ninety-six percent stated an intention to remedy based on the explanations. These are self-reported perceptions and intentions. The experiment does not measure actual reduction in patch time, completion rate, patch quality or reduction in incidents after deployment.

The distinction protects the decision: satisfaction with the explanation is a sign of usability, not operational evidence of security.

A secure contract for the detector

1. Sensor, not authority: the model recommends priority; policy and responsible define blocking, exception and release.
2. Provenance: each alert records package, version, source, digest, moment, features, model and configuration.
3. External evidence: explanations point to artifacts that another person or tool can verify.
4. Inconclusive status: the system distinguishes benign, suspected, confirmed and unverifiable.
5. Baseline by ecosystem: npm and PyPI have distinct patterns; thresholds and error cost need to be measured locally.
6. Defense in depth: signing, lockfile, SBOM, update review, sandbox, SCA, behavior analysis and incident response remain necessary.
7. Drift and revalidation: Performance and explanations are tracked after model, data, and ecosystem changes.
8. Decision receipt: alert, investigation, evidence, exception and authority are linked to the exact release.

What should not become a promise

The paper compares XAI-SCS to existing tools and approaches, but does not establish universal replacement of Snyk, Dependabot, or CVE-driven SCA. The contribution lies in the combination of signs and explanations within the studied scenario. Traditional tools continue to provide inventory, policies, integration and operational intelligence that an isolated experimental model cannot replicate.

Proposals such as federated learning, zero-knowledge proofs and post-quantum enclaves appear as future directions. They have not been validated by experiment and should not be presented as current capabilities.

The authors declare that they have no conflicts of interest. The work received funding from the National Cybersecurity Authority of Saudi Arabia, grant CRPG-25-3096. The article statement informs the use of Grammarly, ChatGPT, Grok and Gemini for language polishing, not for producing scientific results. Until the cutoff of 10/04/2026, we did not find an independent reproduction of the complete study.

Architecture rule

A software chain detector delivers a prioritized hypothesis. An explanation makes this hypothesis more inspectable. Security only appears when the organization connects the signal to verifiable evidence, limited authority, recorded decision, and outcome monitoring. [HARNESS31-C2]

The useful output is not ‘secure package’. It is a package of evidence: what was observed, by what version of the detector, what explanation was produced, what independent verification occurred, what was out of scope, and who authorized the release.

Direct source

1. Explainable artificial intelligence for software supply chain security — Scientific Reports. Article, method, dataset, metrics, developer survey, funding and limitations.

Editorial and responsibility note

This text combines facts attributed to the source with the author's analysis and technical proposal. Personal and professional views are not proven facts; data, denominators, limits and conflicts are indicated when available. The content is informative and does not replace technical, legal, financial or security assessment. Tech Human and Trustyu work commercially on related topics. Research, structure and writing were assisted by AI; Factual review, authorial approval, and publication were confirmed by Fernando Parreiras on 10/4/2026, without additional independent human review.

Research cutoff: 10/04/2026. Editorial status: Special Evidence Brief authorized for 10/7/2026 at 9am BRT.