B
Evidence Grade B
Graded
accuracy verified by the review judge
Safe to Cite
Dated
77.5%
77.5% -- bug fixing success rate for ChatGPT (2023)
The success rate of ChatGPT in bug fixing with follow-up interactions was 77.5%.
- Applies specifically to bug fixing with follow-up interactions on QuixBugs Python benchmark problems
Attribution:
View source
Lighthouse Research Team
Last verified:
·
How we verify data →
Source verified
Evidence graded
Value
77.5%
Unit
Percentage
Data Vintage
2023
Entity
Sample Size
40
Evidence Quality
B
Strong, reliable evidence
Sourced from a well-regarded research firm or industry leader with transparent methodology and a reasonably large sample.
Single source - not yet corroborated
Score breakdown
Extraction
75
Source Trust
60
Citation Chain
20
Independence
30
Methodology
45
Freshness
95
Source
Cornell University researchers (arXiv paper)
Primary
Tier B
https://jarthur.co/chat-gpt-for-coding-strengths-and-weaknesses/ ↗
Source Article
Chat GPT for Coding: Strengths and Weaknesses
“ChatGPT is incredibly limited but good enough at some things to create a misleading impression of greatness. it’s a mistake to be relying on it for anything important right now. it’s a preview of progress; we have lots of work to do on robustness and truthfulness.” – Sam Altman, CEO of OpenAI
Chat …
Original Research
Cornell University researchers (arXiv paper)
· Original study ↗
Methodology
Tested on QuixBugs 40 Python-only problems in an arXiv study from Cornell University researchers
Methodology documented
Segmentation
Industrymarketing
Cite this stat
Plain Text
ChatGPT: 77.5% bug fixing success rate. Source: Cornell University researchers (arXiv paper) (2023). Via Lighthouse Intelligence -- https://lighthousedata.io/data/chatgpt-bug-fixing-success-rate
HTML Embed
<blockquote cite="https://lighthousedata.io/data/chatgpt-bug-fixing-success-rate" style="border-left:3px solid #2563eb;padding:12px 16px;margin:16px 0;font-family:system-ui,sans-serif;background:#f0f4ff;"><strong>77.5%</strong> -- bug fixing success rate for ChatGPT<br><small>Source: Cornell University researchers (arXiv paper) (2023) · <a href="https://lighthousedata.io/data/chatgpt-bug-fixing-success-rate" target="_blank" rel="noopener">Lighthouse Intelligence</a></small></blockquote>
Share
Use this data in your work
Copy the citation, embed the widget, or explore thousands more verified marketing statistics -- all free.