Analysis of LLM Performance on AWS Bedrock: Receipt-item Categorisation Case Study

Dimensions overall score 5.0

• Claude 3.7 Sonnet achieves the most favourable balance between classification accuracy and cost efficiency for receipt-item categorisation.result 90%
• The study systematically evaluates LLMs for receipt-item categorisation within a production-oriented classification framework.method 90%
• The evaluation compares four instruction-tuned models available through AWS Bedrock: Claude 3.7 Sonnet, Claude 4 Sonnet, Mixtral 8x7B Instruct, and Mistral 7B Instruct.method 90%
• The study assesses model performance across accuracy, response stability, and token-level cost.method 90%
• The study investigates what prompting methods (zero-shot or few-shot) are especially appropriate in terms of both accuracy and incurred costs.method 90%
• The evaluation is cost-aware, considering token-level costs in its assessment.method 85%
• The study focuses on a specific application: receipt-item categorisation.technical 90%