Links table
Abstract and 1. Introduction
2. Methodology and 2.1. Search questions
2.2. Collection
2.3. Place a sign of data
2.4. Data extraction
2.5. Data analysis
3. Results and interpretation and 3.1. Type of problems (RQ1)
3.2. Type of causes (RQ2)
3.3. Solutions type (RQ3)
4. The consequences
4.1. The effects of Copilot users
4.2. The effects of Copilot team
4.3. The effects of researchers
5. Justice threats
6. Related work
6.1. Evaluating the quality of code created by Copilot
6.2. Copilot effect on practical development and 6.3. Conclusive summary
7. Conclusions, data, approval, a statement of credit and references contribution
4.3. The effects of researchers
The use of Copilot may change the coding process and increase the cost of time for code suggestions, making explanation of the software instructions very important. In our research, incomprehensible suggestions are ranked fourth most common to the suggestion content, and the feature of explaining the code is also requested. Some users have mentioned that the code created by COPILOT is very long, which leads to a decrease in reading. This indicates that when Copilot provides relatively complicated suggestions, or when users lack a coding experience in a specific field, understanding the logic of code and verifying its authenticity can take a long time. In this case, users may refrain from adopting Copilot’s code suggestions because they make them feel lost control of the coding task, according to the Vaithiltam Et al. (2022). The study by Wang and others. (2023B) also shows that the use of the created code of artificial intelligence can lead to a great review pressure. Therefore, we believe that artificial intelligence coding tools such as Copilot will change the customization of the time you spend in various tasks in developing software. As expected by some users, the feature of explaining the code will be useful. GitHub Copilot Chat is now generally available for institutions and individuals (Zhao, 2023), which is run by GPT-4. COPILOT feature enables more flexible interaction with users, providing the ability to implement functions such as explaining software instructions, rewriting software instructions, and discovering weakness. For future research, we plan to explore how the chat feature affects the programming process and enhance the efficiency of coding tasks.
To accurately measuring user satisfaction with Copilot, it is necessary to consider programming tasks across various fields of applications, as well as the purposes that users use from Copilot. We have identified relatively few problems in the suggestion content, and possible reasons is that users are less inclined to report specific problems related to the state for public questions and answers, indicating a certain degree of tolerance with wrong suggestions of artificial intelligence (Weisz et al., 2021). Therefore, the results of our study may not fully reflect the satisfaction of the code created by Copilot. Users often suggest new feature requests based on their individual needs and coding habits, as well as their experience in using Copilot. Some of these requests are closely related to the fields of application developed by COPILOT users, such as the development of the front web and the development of the game. Thus, user satisfaction levels may vary with COPILOT across the various application areas. Thus, the evaluation of COPILOT capabilities through the lens of individual programming tasks, such as treating algorithm problems or writing SQL phrases, may not represent actual user satisfaction in other application areas. Moreover, our previous research found that users have various goals when using Copilot (Zhang et al., 2023), which directly affects their expectations. For example, users who are looking to quickly understand new technologies with the help of Copilot are more interested in whether code suggestions provide useful instructions for programming for subsequent steps; On the contrary, those who want to use Copilot for frequent coding tasks, such as CRUD operations in the database, are more interested in whether Copilot can create the correct software instructions to reduce the great effort that you spend on fixing similar problems in code suggestions. Therefore, the intended use of Copilot also stands as a decisive factor that affects user satisfaction with the Scomon. A focused study on user satisfaction on COPILOT will provide an insightful assessment.
Authors:
(1) Xu Zhou, College of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(2) Ping Liang (author), College of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(3) Becky Chang, College of Computer Science, Wuhan University, Wuhan, China ([email protected]);
(4) Zengyang Li, College of Computer Science, Central China University, Wuhan, China ([email protected]);
(5) Aakash Ahmed, College of Computing and Communications, University of Lancaster Leipzig, Leipzig, Germany ([email protected]);
(6) Mojtaba Shahin, College of Computing Technologies, RMIT University, Melbourne, Australia ([email protected]);
(7) Mohamed Wasim, College of Information Technology, University of Gifksel, Jevskil, Finland ([email protected]).
