LLM-Powered Flood Depth Estimation from Social Media Imagery: A Vision-Language Model Framework with Mechanistic Interpretability for Transportation Resilience

Dimensions overall score 8.0

• FloodLlama, a fine-tuned open-source vision-language model (VLM), can estimate continuous flood depth from single street-level images.method 95%
  Evidencepartial

  This study presents FloodLlama, a fine-tuned open-source vision-language model (VLM) for continuous flood depth estimation from single street-level images

  Implicationpartial

  This is a core claim explicitly stated in the abstract describing the main contribution of the paper.

  Verificationpartial

  partial

• A synthetic dataset of approximately 190,000 images was generated for training, covering seven vehicle types, four weather conditions, and 41 depth levels (0-40 cm at 1 cm resolution).method 95%
  Evidencepartial

  A synthetic dataset of approximately 190000 images was generated, covering seven vehicle types, four weather conditions, and 41 depth levels (0-40 cm at 1 cm resolution).

  Implicationpartial

  The abstract provides specific details about the synthetic dataset's size, content, and resolution.

  Verificationpartial

  partial

• Chain-of-thought (CoT) reasoning improves flood depth estimation performance at greater depths compared to simple prompts.result 90%
  Evidencepartial

  Evaluation across 34797 trials reveals a depth-dependent prompt effect: simple prompts perform better for shallow flooding, whereas chain-of-thought (CoT) reasoning improves performance at greater depths.

  Implicationpartial

  The abstract directly states this finding with a clear comparison between prompt types and depth levels.

  Verificationpartial

  partial

• FloodLlama achieves a mean absolute error (MAE) below 0.97 cm for deep flooding and above 96.8% accuracy for shallow depths.result 95%
  Evidencepartial

  FloodLlama achieves a mean absolute error (MAE) below 0.97 cm and Acc@5cm above 93.7% for deep flooding, exceeding 96.8% for shallow depths.

  Implicationpartial

  Specific quantitative performance metrics (MAE and Acc@5cm) are provided for different flooding scenarios.

  Verificationpartial

  partial

• A five-phase mechanistic interpretability framework identifies layer L23 as the critical depth-encoding transition.technical 90%
  Evidencepartial

  A five-phase mechanistic interpretability framework identifies layer L23 as the critical depth-encoding transition

  Implicationpartial

  The abstract clearly states the outcome of the interpretability framework and identifies a specific layer.

  Verificationpartial

  partial

• Selective fine-tuning using the interpretability framework reduces trainable parameters by 76-80% while maintaining accuracy.technical 90%
  Evidencepartial

  and enables selective fine-tuning that reduces trainable parameters by 76-80% while maintaining accuracy.

  Implicationpartial

  The abstract quantifies the reduction in trainable parameters and states that accuracy is maintained.

  Verificationpartial

  partial

• The Tier 3 configuration of FloodLlama achieves 98.62% accuracy on real-world data and demonstrates strong robustness under visual occlusion.result 95%
  Evidencepartial

  The Tier 3 configuration achieves 98.62% accuracy on real-world data and shows strong robustness under visual occlusion.

  Implicationpartial

  Specific performance on real-world data and a key robustness characteristic are provided.

  Verificationpartial

  partial

• A TikTok-based data pipeline, validated on 676 annotated flood frames from Detroit, demonstrates the feasibility of real-time, crowd-sourced flood sensing.method 95%
  Evidencepartial

  A TikTok-based data pipeline, validated on 676 annotated flood frames from Detroit, demonstrates the feasibility of real-time, crowd-sourced flood sensing.

  Implicationpartial

  The abstract describes the validation of a specific data pipeline with quantitative evidence and its demonstrated feasibility.

  Verificationpartial

  partial

Competitive landscape

FloodLlama is an open-source vision-language model for real-time flood depth estimation from social media imagery, enhancing transportation resilience.