A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)

Code for ACL 2024 paper "TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space"

[ICML'2024] Can AI Assistants Know What They Don't Know?

A toolbox for benchmarking Multimodal LLM Agents trustworthiness across truthfulness, controllability, safety and privacy dimensions through 34 interactive tasks

Unofficial implementation of Microsoft’s Claimify Paper: extracts specific, verifiable, decontextualized claims from LLM Q&A to be used for Hallucination, Groundedness, Relevancy and Truthfulness detection

Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency

Google Chrome Extension to add LLM text reading in a browser

Behavioral auditing & repair toolkit for LLMs. Measures 8 dimensions via confidence probes. Rho-Surgery fixes miscalibration with contrastive LoRA + gamma protection. CatSAE for feature-level intervention. SVD compression with knowledge preservation.

Improving LLM truthfulness via reporting confidence

A research project on competing notions of truth in large language models.

Example VerityIndex Reports on Media

My first eval / interpretability project. Testing steering vectors for truthfulness and honesty on TruthfulQA and MASK.