Sustainability Is Not Linear: Quantifying Performance, Energy, and Privacy Trade-offs in On-Device Intelligence

Dimensions overall score 4.0

• A reproducible experimental pipeline was constructed to profile the interplay between energy consumption, latency, and quality of on-device LLMs.method 95%
  Evidencepartial

  To navigate this landscape, we constructed a reproducible experimental pipeline to profile the complex interplay between energy consumption, latency, and quality.

  Implicationpartial

  The abstract explicitly states the construction of this pipeline and its purpose.

  Verificationpartial

  partial

• Modern importance-aware quantization yields negligible energy savings compared to standard mixed-precision methods for LLMs on mobile devices.result 90%
  Evidencepartial

  Our investigation uncovered a counter-intuitive quantization-energy paradox. While modern importance-aware quantization successfully reduces memory footprints to fit larger models into RAM, we found it yields negligible energy savings compared to standard mixed-precision methods.

  Implicationpartial

  The abstract clearly states this counter-intuitive finding and its implication.

  Verificationpartial

  partial

• For battery life on mobile devices, the architecture of an LLM is a more decisive factor than its quantization scheme.result 90%
  Evidencepartial

  This proves that for battery life, the architecture of the model, not its quantization scheme, is the decisive factor.

  Implicationpartial

  This is a direct conclusion drawn from the quantization-energy paradox finding in the abstract.

  Verificationpartial

  partial

• Mixture-of-Experts (MoE) architectures offer the storage capacity of a 7B model while maintaining the lower energy profile of a 1B to 2B model.result 95%
  Evidencepartial

  We further identified that Mixture-of-Experts (MoE) architectures defy the standard size-energy trend, offering the storage capacity of a 7B model while maintaining the lower energy profile of a 1B to 2B model.

  Implicationpartial

  The abstract explicitly highlights this characteristic of MoE architectures.

  Verificationpartial

  partial

• Mid-sized LLMs, such as Qwen2.5-3B, represent a pragmatic sweet spot balancing response quality with sustainable energy consumption on-device.result 90%
  Evidencepartial

  Finally, an analysis of these multi-objective trade-offs reveals a pragmatic sweet spot of mid-sized models, such as Qwen2.5-3B, that effectively balance response quality with sustainable energy consumption.

  Implicationpartial

  The abstract concludes with this practical recommendation based on the trade-off analysis.

  Verificationpartial

  partial

• The experimental pipeline captured granular power metrics across eight LLMs ranging from 0.5B to 9B parameters without requiring root access.method 95%
  Evidencepartial

  Unlike theoretical studies, we captured granular power metrics across eight models ranging from 0.5B to 9B parameters without requiring root access, ensuring our findings reflect realistic user conditions.

  Implicationpartial

  The abstract specifies the range of models and the condition of not requiring root access.

  Verificationpartial

  partial

• The study utilized the llama.cpp inference engine for executing LLMs on the Android device.method 90%
  Evidencepartial

  All selected models were executed using the llama.cpp [8] inference engine, a highly optimized framework designed specifically for resource-constrained edge environments.

  Implicationpartial

  The methodology section explicitly mentions the use of llama.cpp.

  Verificationpartial

  partial

Competitive landscape

This research quantifies the complex trade-offs between performance, energy, and privacy for running LLMs on edge devices, revealing counter-intuitive findings about quantization and model architecture.