ARXIV:2602.14357 · LLM DESIGN AND EVALUATION · SUBMITTED 02 APR · 02:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

arXiv

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Blocked on Code›Score3.0Evidence unverified

Opportunity summary

Pain An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Evidence 0 refs | 0 sources | 17% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise. This paper examines the challenges and trade-offs in LLM development through a…

METHOD

Full abstract

Large Language Models (LLMs) are increasingly developed for use in complex professional domains, yet little is known about how teams design and evaluate these systems in practice. This paper examines the challenges and trade-offs in LLM development through a 12-week ethnographic study of a team building a pedagogical chatbot. The researcher observed design and evaluation activities and conducted interviews with both developers and domain experts. Analysis revealed four key practices: creating workarounds for data collection, turning to augmentation when expert input was limited, co-developing evaluation criteria with experts, and adopting hybrid expert-developer-LLM evaluation strategies. These practices show how teams made strategic decisions under constraints and demonstrate the central role of domain expertise in shaping the system. Challenges included expert motivation and trust, difficulties structuring participatory design, and questions around ownership and integration of expert knowledge. We propose design opportunities for future LLM development workflows that emphasize AI literacy, transparent consent, and frameworks recognizing evolving expert roles.

RESULT

ScienceToStartup currently rates this 3.0/10 on the public viability pass. These practices show how teams made strategic decisions under constraints and demonstrate the central role of domain expertise in shaping the system.

WHY NOW

LLM Design and Evaluation moved forward this cycle; last verified April 2026. Public score 3.0/10.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score3.0

PainAn ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Evidence0 refs | 0 sources | 17% coverage

Blockermissing authors

Analysis summary

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

ARXIV:2602.14357 · LLM DESIGN AND EVALUATION · SUBMITTED 02 APR · 02:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

arXiv

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Blocked on Code›Score3.0Evidence unverified

Opportunity summary

Pain An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Evidence 0 refs | 0 sources | 17% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

METHOD

Full abstract

RESULT

WHY NOW

LLM Design and Evaluation moved forward this cycle; last verified April 2026. Public score 3.0/10.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score3.0

PainAn ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Evidence0 refs | 0 sources | 17% coverage

Blockermissing authors

Analysis summary

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Paper Pack

10.48550/arXiv.2602.14357

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Abstract

Source availability

PDF linked

The paper record includes a public PDF URL.

Extraction status

Derived fallback

Read summaries are estimated from adjacent metadata, not verified extraction rows.

Proof status

unverified

0 refs; 0 sources; 17% coverage.

What was readable

linkedon filenot materializedderived fallback46 indexednot indexed

Derived fallback: Estimated from adjacent evidence; not verified from source.

Viability

3.0

Time to MVP

MVP estimate missing

Commercial

No commercial flags on file

Export

Preparing verified analysis

lens / founder

PROBLEM

METHOD

RESULT

WHY NOW

LLM Design and Evaluation moved forward this cycle; last verified April 2026. Public score 3.0/10.

Claim map

Abstract-backed public claims while anchored extraction refreshes.

Strong 0Mixed 0Weak 4

Evidencepartial
An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise. This paper examines the challenges and trade-offs in LLM development through a 12-week ethnographic study of a team building a pedagogical chatbot.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial
Evidencepartial
Large Language Models (LLMs) are increasingly developed for use in complex professional domains, yet little is known about how teams design and evaluate these systems in practice. This paper examines the challenges and trade-offs in LLM development through a 12-week ethnographic study of a team building a pedagogical chatbot.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial
Evidencepartial
ScienceToStartup currently rates this 3.0/10 on the public viability pass. These practices show how teams made strategic decisions under constraints and demonstrate the central role of domain expertise in shaping the system.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial
Evidencepartial
LLM Design and Evaluation moved forward this cycle; last verified April 2026. Public score 3.0/10.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial

Constellation map

Paper-native neighborhood for concepts, methods, materials, markets, and competitors. Missing lanes stay labeled instead of disappearing behind commercialization gates.

Open full Signal Canvas

Concepts

not indexed

Methods

Materials

PDF linked

Markets

LLM Design and Evaluation

Competitors

not indexed

Competitive landscape

An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.

Segment

LLM Design and Evaluation

Adoption evidence

No public code link in the paper record yet

Commercial read

3.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

Buzz

No indexed public discussion is attached to 2602.14357 yet. That is a visibility signal, not a blank module: the monitor is watching the public channels below.

Hacker News

Not indexed yet

Bluesky

Not indexed yet

PDF

Preview the source document here, or use the hero PDF action for a new tab.

References(46)

Participatory Design in Human-Computer Interaction: Cases, Characteristics, and Lessons

2025Xiang Qi, Junnan Yu

A Survey on LLM-as-a-Judge

2024Jiawei Gu, Xuhui Jiang et al.

Developer Challenges on Large Language Models: A Study of Stack Overflow and OpenAI Developer Forum Posts

2024Khairul Alam, Kartik Mittal et al.

Development of LLM-Based System for IT Talent Interview

2024David K. H. Pasaribu, Agung Dewandaru et al.

PromptHive: Bringing Subject Matter Experts Back to the Forefront with Collaborative Prompt Engineering for Educational Content Creation

2024Mohi Reza, Ioannis Anastasopoulos et al.

Understanding LLM Development Through Longitudinal Study: Insights from the Open Ko-LLM Leaderboard

2024Chanjun Park, Hyeonwoo Kim

An Empirical Study on Challenges for LLM Application Developers

2024Xiang Chen, Chaoyang Gao et al.

Are LLMs Correctly Integrated into Software Systems?

2024Yuchen Shao, Yuheng Huang et al.

Human-Centered Design Recommendations for LLM-as-a-judge

2024Qian Pan, Zahra Ashktorab et al.

Mediating Culture: Cultivating Socio-cultural Understanding of AI in Children through Participatory Design

2024Aayushi Dangol, Michele Newman et al.

A Survey of Useful LLM Evaluation

2024Ji-Lun Peng, Sijia Cheng et al.

Integrating Expertise in LLMs: Crafting a Customized Nutrition Assistant with Refined Template Instructions

2024Annalisa Szymanski, Brianna L Wimer et al.

Efficient LLM Comparative Assessment: A Product of Experts Framework for Pairwise Comparisons

2024Adian Liusie, Vatsal Raina et al.

Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences

2024Shreya Shankar, J.D. Zamfirescu-Pereira et al.

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

2024Yann Dubois, Bal'azs Galambosi et al.

EvaluLLM: LLM assisted evaluation of generative outputs

2024Michael Desmond, Zahra Ashktorab et al.

From Prompt Engineering to Prompt Science with Humans in the Loop

2024Chirag Shah

The Rise of AI Art: A Look Through Digital Artists' Eyes

2024Lixuan Zhang, Katelyn Wilson et al.

ChainForge: An open-source visual programming environment for prompt engineering

2023Ian Arawjo, Priyan Vaithilingam et al.

Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

2023Xiangyu Qi, Yi Zeng et al.

Showing 20 of 46 references

CITED BY

No citing papers are indexed in the public S2S graph yet. This is an explicit zero-signal state, not a hidden lookup.

Foundation

Prior WorkThe Crutch or the Ceiling? How Different Generations of LLMs Shape EFL Student Writings

3.0

Extension

Builds On ThisModularizing Educational LLM-Agency for Fostering Responsible Learning Assistance

2.0

Builds On ThisPersuadability and LLMs as Legal Decision Tools

2.0

Commercially relevant

Higher ViabilityEvaluating LLMs for Answering Student Questions in Introductory Programming Courses

7.0

Higher ViabilityLLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation

7.0

Higher ViabilityHow Far Are We? Systematic Evaluation of LLMs vs. Human Experts in Mathematical Contest in Modeling

4.0

Higher ViabilityFrom Untamed Black Box to Interpretable Pedagogical Orchestration: The Ensemble of Specialized LLMs Architecture for Adaptive Tutoring

7.0

Higher ViabilityHuman-Centered Evaluation of an LLM-Based Process Modeling Copilot: A Mixed-Methods Study with Domain Experts

6.0

Higher ViabilityLanguage Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts

5.0

Higher Viability"How Do I ...?": Procedural Questions Predominate Student-LLM Chatbot Conversations

5.0

Conflicting

none indexed

Owned Distribution

Subscribe to the weekly brief

Get the weekly shortlist of commercializable papers, benchmark movers, and proof receipts that matter for product execution.

Agent drawer

5 surfaces preserved for agents. Humans can ignore.

Developer contracts, payload previews, evidence maps, and run controls stay here instead of the Read, Build, and Track workspace.

Run context

Paper: 2602.14357
Route: /paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study
Active tab: read
Artifact: key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

Available agents

Read extractor
Build planner
Track monitor
Competitive mapper
Related-paper scout

API/MCP endpoints

REST paper pack API/api/v1/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study/paper-pack
REST build passport API/api/v1/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study/build-passport
REST OpenAPI/api/openapi.json
MCP descriptor/api/mcp
MCP resourcesciencetostartup://surfaces/paper-workspace

Tool contracts

paper_packbuild_passportopportunity_kernelforesightsource_proofevidence_state

Payload preview

Inspect payload

{
  "contract_version": "paper-r2",
  "paper_id": "ebd1f236-419b-4c6b-99bc-8c8b22b78c8a",
  "arxiv_id": "2602.14357",
  "canonical_route": "/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study",
  "active_tab": "synced from current hash by the drawer client",
  "selected_artifact": "key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study",
  "endpoints": {
    "paper_pack": "/api/v1/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study/paper-pack",
    "build_passport": "/api/v1/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study/build-passport",
    "mcp_resource": "sciencetostartup://surfaces/paper-workspace"
  }
}

Schema validation

paper-r2 contract: present
JSON-LD twin: SSR emitted
OpenAPI path parity: /api/openapi.json
MCP resource parity: paper-workspace

Job trace

queued: drawer opened by user action
running: inspect or copy payload
succeeded: payload available in SSR
failed: route errors appear in evidence cards

Evidence map

sources used: page freshness, source proof anchors, JSON-LD
missing sources: exposed by PaperPack and EvidenceState chips
derived fallbacks: marked unverified before handoff

Page Freshness

Canonical route, proof status, last verified, refs, sources, and coverage.

Page Freshness

Paper proof surface

Canonical route: /paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

stale

Proof freshness: stale
Proof status: unverified
Display score: 3/10
Last proof check: 2026-04-02
Score updated: 2026-04-02
Score fresh until: 2026-05-02
References: 0
Source count: 0
Coverage: 17%

This page is showing the last landed evidence receipt and score bundle because the latest proof data is outside the freshness window.

OpenAlex: pending — this preprint is not yet indexed by OpenAlex.

Agent Handoff

Endpoint list, payload shape, route context, and copyable handoff data.

Agent Handoff

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

Canonical ID key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study | Route /paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

REST example

curl https://sciencetostartup.com/api/v1/agent-handoff/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

MCP example

{
  "tool": "get_paper",
  "arguments": {
    "arxiv_id": "2602.14357"
  }
}

source_context

{
  "surface": "paper",
  "mode": "paper",
  "query": "Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study",
  "normalized_query": "2602.14357",
  "route": "/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study",
  "paper_ref": "key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study",
  "topic_slug": null,
  "benchmark_ref": null,
  "dataset_ref": null
}

Buildability Receipt

Verdict, compute envelope, blockers, signature state, and receipt links.

Paper proof page receipt window

Not build-ready: Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

/buildability/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

Ignoreblocked

Subject: Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

Verdict

Ignore

Verdict is Ignore because current viability and proof state do not clear the buildability gate.

Time to first demo

Insufficient data

No first-demo timestamp, owner estimate, or elapsed demo receipt is attached to this surface.

Compute envelope

Structured compute envelope

Insufficient data

No data, compute, hardware, memory, latency, dependency, or serving requirement receipt is attached.

Evidence ids

Receipt path

/buildability/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

Paper ref

key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study

arXiv id

2602.14357

Freshness

Generated at

2026-04-02T02:30:40.136Z

Evidence freshness

stale

Last verification

2026-04-02T02:30:40.136Z

Sources

References

Coverage

17%

Hash state

Lineage hash

b8df24ac8b4a7abf863ab734e1378e33bae0a83756e057bc62f2266d590b1639

Canonical opportunity-kernel lineage hash.

Signature state

External signature

unsigned_external

No founder, registry, pilot, or production-adoption signature is attached to this receipt.

Verification

not_verified

Verification is blocked until an external signature is provided.

Blockers

Missing: repo_url
Missing: references
Missing: proof_status
Missing: distribution_readiness_scores
Missing: paper_extraction_scorecards
Unknown: distribution readiness has not been computed yet
Unknown: proof verification has not been recorded yet

Verification pending / evidence receipt incomplete

repo_url

references

Missing proof, requirement, signature, approval, adoption, or telemetry fields are blockers and must not be inferred.

Open receipt API receipt Build Loop Signal Canvas Proof divergence Divergence API Brier outcomes API

Source Proof anchors

Visual citations from the paper document graph.

JSON-LD twin

The application/ld+json payload rendered for agents.

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "WebPage",
      "@id": "https://sciencetostartup.com/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study#webpage",
      "url": "https://sciencetostartup.com/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study",
      "name": "Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study",
      "description": "An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.",
      "isPartOf": {
        "@id": "https://sciencetostartup.com/#website"
      }
    },
    {
      "@type": "ScholarlyArticle",
      "@id": "https://sciencetostartup.com/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study#scholarlyArticle",
      "headline": "Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study",
      "description": "An ethnographic study revealing key practices and challenges in LLM design and evaluation in professional domains with a focus on domain expertise.",
      "url": "https://sciencetostartup.com/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study",
      "sameAs": "https://arxiv.org/abs/2602.14357",
      "identifier": {
        "@type": "PropertyValue",
        "propertyID": "arXiv",
        "value": "2602.14357"
      },
      "isAccessibleForFree": true,
      "isPartOf": {
        "@id": "https://sciencetostartup.com/#website"
      },
      "datePublished": "2026-02-16T00:21:58.000Z",
      "citation": [
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "3431a3387314911f93bdb07ac8387dabbd5eaadb"
          },
          "url": "https://www.semanticscholar.org/paper/3431a3387314911f93bdb07ac8387dabbd5eaadb"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "e24424283c02fbe7f641e5b3490d7bb059f8355a"
          },
          "url": "https://www.semanticscholar.org/paper/e24424283c02fbe7f641e5b3490d7bb059f8355a"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "e56df5dae702974700babab8935ee543ec204942"
          },
          "url": "https://www.semanticscholar.org/paper/e56df5dae702974700babab8935ee543ec204942"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "f3fffadbec3bc3b6c29d7bff55f60be491ab8379"
          },
          "url": "https://www.semanticscholar.org/paper/f3fffadbec3bc3b6c29d7bff55f60be491ab8379"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "8b5997fc40ba15ad93a032284ee99ebe2a5fa6ee"
          },
          "url": "https://www.semanticscholar.org/paper/8b5997fc40ba15ad93a032284ee99ebe2a5fa6ee"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "af9e65570373e6eff074e0d9708a38741c2a8684"
          },
          "url": "https://www.semanticscholar.org/paper/af9e65570373e6eff074e0d9708a38741c2a8684"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "a0b720bc1f7e15594ab5b96819f2a5f904c02da5"
          },
          "url": "https://www.semanticscholar.org/paper/a0b720bc1f7e15594ab5b96819f2a5f904c02da5"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "e9661585d0efc02b4c941e5398ab6ef85063c9c0"
          },
          "url": "https://www.semanticscholar.org/paper/e9661585d0efc02b4c941e5398ab6ef85063c9c0"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "52570796ad1fdc20a367caf1d099e729d6456241"
          },
          "url": "https://www.semanticscholar.org/paper/52570796ad1fdc20a367caf1d099e729d6456241"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "044a6902f6b3751f37af5eef5d99855121bf68da"
          },
          "url": "https://www.semanticscholar.org/paper/044a6902f6b3751f37af5eef5d99855121bf68da"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "4906654ecb2d42cfee13f46bebe2ec02bb6442b1"
          },
          "url": "https://www.semanticscholar.org/paper/4906654ecb2d42cfee13f46bebe2ec02bb6442b1"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "333df263d68afa35867286cde555de5e0629722e"
          },
          "url": "https://www.semanticscholar.org/paper/333df263d68afa35867286cde555de5e0629722e"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "4a70c45876cfc39a740ba9851e84608d10921660"
          },
          "url": "https://www.semanticscholar.org/paper/4a70c45876cfc39a740ba9851e84608d10921660"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "6098ac103ba222f9c3b089714ef3100357993255"
          },
          "url": "https://www.semanticscholar.org/paper/6098ac103ba222f9c3b089714ef3100357993255"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "eb375712bd37250c350ecd3f559e1879e87eb3e5"
          },
          "url": "https://www.semanticscholar.org/paper/eb375712bd37250c350ecd3f559e1879e87eb3e5"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "1869b63cbb1670938fa21670021e405d5dd40a48"
          },
          "url": "https://www.semanticscholar.org/paper/1869b63cbb1670938fa21670021e405d5dd40a48"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "8396c663cae6727bfa7d81b6a16e055bdf47e1b0"
          },
          "url": "https://www.semanticscholar.org/paper/8396c663cae6727bfa7d81b6a16e055bdf47e1b0"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "bb411609bbe11aa674e484507c989ad933b1e64c"
          },
          "url": "https://www.semanticscholar.org/paper/bb411609bbe11aa674e484507c989ad933b1e64c"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "0e0e706e13f160e74cac9556f28ab9a358c148d2"
          },
          "url": "https://www.semanticscholar.org/paper/0e0e706e13f160e74cac9556f28ab9a358c148d2"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "be2b0125f3161739c685e9f86d9fd49f9f6d99c8"
          },
          "url": "https://www.semanticscholar.org/paper/be2b0125f3161739c685e9f86d9fd49f9f6d99c8"
        }
      ],
      "additionalProperty": [
        {
          "@type": "PropertyValue",
          "propertyID": "viabilityScore",
          "value": 3
        },
        {
          "@type": "PropertyValue",
          "propertyID": "researchDomain",
          "value": "LLM Design and Evaluation"
        }
      ]
    },
    {
      "@type": "BreadcrumbList",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://sciencetostartup.com"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "LLM Design and Evaluation",
          "item": "https://sciencetostartup.com/topics"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "Key Considerations for Domain Expert Involvement in LLM Desi",
          "item": "https://sciencetostartup.com/paper/key-considerations-for-domain-expert-involvement-in-llm-design-and-evaluation-an-ethnographic-study"
        }
      ]
    }
  ]
}

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study

Claim map

Constellation map

Competitive landscape

Buzz

PDF

References(46)

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

References(46)

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline