arbiter
JournalsPricingAccount

Complex questions and quality answers: Comparing ChatGPT and Gemini as research collaborators

Benhur Ravuri et al.

Journal of the Association for Information Science and Technology (JASIST)2026https://doi.org/10.1002/asi.70056article
AJG 3
Weight
0.50

What the paper says

AI chatbots are increasingly popular, but how they handle complex questions and how this affects the quality of their answers remains underexplored. This study examined whether chatbots such as ChatGPT and Gemini provide high‐quality answers to users' questions. To determine whether LLMs provided accurate, complete responses and support for further assistance, and addressed different difficulty levels and question types, we used ChatGPT 4o‐mini and Gemini 1.5 Flash to analyze 84 authentic library reference questions of varying complexity and types. Our analyses demonstrated a strong, statistically significant association between question complexity (READ) levels and further assistance. ChatGPT4o‐mini suggests that as complexity increases, it provides more resources but still fails to give a complete answer, whereas Gemini 1.5 Flash also reflected a significant association between question type and completeness. We conclude that, compared with ChatGPT 4o‐mini, Gemini 1.5 Flash is sensitive to all question types, suggesting it can provide more consistently high‐quality answers. These findings suggest that understanding the relationship between question complexity and answer quality can optimize LLMs for better information seeking. As LLMs are continually updating, this study used ChatGPT‐4o‐mini and Gemini 1.5 Flash. Future research should evaluate newer LLMs and human responses using a comparative methodology.

Open paper page →

Cite this paper

https://doi.org/https://doi.org/10.1002/asi.70056

Or copy a formatted citation

@article{benhur2026,
  title        = {{Complex questions and quality answers: Comparing ChatGPT and Gemini as research collaborators}},
  author       = {Benhur Ravuri et al.},
  journal      = {Journal of the Association for Information Science and Technology (JASIST)},
  year         = {2026},
  doi          = {https://doi.org/https://doi.org/10.1002/asi.70056},
}

Paste directly into BibTeX, Zotero, or your reference manager.

Flag this paper

Complex questions and quality answers: Comparing ChatGPT and Gemini as research collaborators

Flags are reviewed by the Arbiter methodology team within 5 business days.

Evidence weight

0.50

Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40

F · citation impact0.50 × 0.4 = 0.20
M · momentum0.50 × 0.15 = 0.07
V · venue signal0.50 × 0.05 = 0.03
R · text relevance †0.50 × 0.4 = 0.20

† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.