Dimensions overall score 7.0

GitHub Code Pulse

No public code linked for this paper yet.

• CREval is a fully automated question-answer (QA)-based evaluation pipeline for creative image manipulation.method 95%
• CREval-Bench is a benchmark designed for creative image manipulation under complex instructions, comprising over 800 editing samples and 13K evaluation queries.method 95%
• Closed-source models generally outperform open-source models on complex and creative image manipulation tasks.result 90%
• All evaluated models still struggle to effectively complete complex and creative image editing tasks.limitation 90%
• User studies demonstrate strong consistency between CREval's automated metrics and human judgments.method 95%
• Current generative image generation and editing models face significant challenges in handling complex instruction-based tasks, particularly in 'free-style creative image editing' scenarios.limitation 90%
• CREval evaluates edited images across three metrics: Instruction Following (IF), Visual Consistency (VC), and Visual Quality (VQ).method 95%
• CREval is a fully automated question-answer (QA)-based evaluation pipeline for creative image manipulation.method 95%
• CREval-Bench is a benchmark designed for creative image manipulation under complex instructions, comprising over 800 editing samples and 13K evaluation queries.method 95%
• Closed-source models generally outperform open-source models on complex and creative image manipulation tasks.result 90%
• All evaluated image manipulation models struggle to effectively complete complex and creative editing tasks.limitation 90%
• User studies demonstrate strong consistency between CREval's automated metrics and human judgments.method 95%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card