Academia.eduAcademia.edu

Figure 3 – uploaded by WARSE The World Academy of Research in Science and Engineering

See full PDFdownloadDownload figure
b) Sample Response

Figure 3 b) Sample Response

Related Figures (6)

uses a special multi-layer transformer network architecture to process and generate human-like conversational dialogue [10]. ChatGPT has continuously evolved since 2018 through numerous updates and enhancements. It works through a transformer [33] which uses specialized algorithms that can recognize long-range patterns in sequences of data.
uses a special multi-layer transformer network architecture to process and generate human-like conversational dialogue [10]. ChatGPT has continuously evolved since 2018 through numerous updates and enhancements. It works through a transformer [33] which uses specialized algorithms that can recognize long-range patterns in sequences of data.
Table 1: Summary Description of dataset comprising problem category, difficulty level, and the problem IDs in LeetCode.
Table 1: Summary Description of dataset comprising problem category, difficulty level, and the problem IDs in LeetCode.
To address the limitations of the previous approach, we applied another approach. In this approach, we replicated the problem description along with the test cases and constraints. We actually presented the whole problem statement as input to ChatGPT. This approach can bypass the knowledge limitation of ChatGPT to access the problem statements included after September 2021. ChatGPT could respond well with a generated coding solution
To address the limitations of the previous approach, we applied another approach. In this approach, we replicated the problem description along with the test cases and constraints. We actually presented the whole problem statement as input to ChatGPT. This approach can bypass the knowledge limitation of ChatGPT to access the problem statements included after September 2021. ChatGPT could respond well with a generated coding solution
Figure 5 : Comparison between the acceptance rate of human participants and ChatGPT varying from the difficulty level of problem statements. for the problems under the Binary Search category with the highest acceptance rate of 83.33%. In contrast, it obtained the lowest acceptance rate in solving problems using greedy algorithm, which stands at 50.00%. This indicates ChatGPT struggled most in Greedy Approach category. The submission status of ChatGPT generated responses across different categories are illustrated in Figure 6 which reflects that ChatGPT got the Wrong Answer verdict in 50.00% cases.
Figure 5 : Comparison between the acceptance rate of human participants and ChatGPT varying from the difficulty level of problem statements. for the problems under the Binary Search category with the highest acceptance rate of 83.33%. In contrast, it obtained the lowest acceptance rate in solving problems using greedy algorithm, which stands at 50.00%. This indicates ChatGPT struggled most in Greedy Approach category. The submission status of ChatGPT generated responses across different categories are illustrated in Figure 6 which reflects that ChatGPT got the Wrong Answer verdict in 50.00% cases.
Figure 6 : Submission status of ChatGPT generated responses across different categories Conversely, ChatGPT performs worse compared to average human performance in Math and Bit Manipulation. Problems under advanced topics like these tend to be mostly attempted by mid-level or expert participants who possess deep knowledge of programming and problem-solving skills. Consequently, human submissions in these categories are relatively fewer compared to the basic topics and most of them are of higher accuracy due to the expertise of the participants. Therefore, the contrast between the submission volume and accepted submissions can potentially be a reason that human acceptance rates are higher than ChatGPT in these categories. These insights suggest that continued research and further improvements are needed to bridge the performance gap of ChatGPT to solve problems under complex problem categories and enhance its understanding of complex mathematical operations.
Figure 6 : Submission status of ChatGPT generated responses across different categories Conversely, ChatGPT performs worse compared to average human performance in Math and Bit Manipulation. Problems under advanced topics like these tend to be mostly attempted by mid-level or expert participants who possess deep knowledge of programming and problem-solving skills. Consequently, human submissions in these categories are relatively fewer compared to the basic topics and most of them are of higher accuracy due to the expertise of the participants. Therefore, the contrast between the submission volume and accepted submissions can potentially be a reason that human acceptance rates are higher than ChatGPT in these categories. These insights suggest that continued research and further improvements are needed to bridge the performance gap of ChatGPT to solve problems under complex problem categories and enhance its understanding of complex mathematical operations.
Related topics:
Engineering
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved