Artificial intelligence models, refinement to better get to know sex and race in stories
Authors:
(1) Ivan Shih, the Association of Youth Data Scholars ([email protected]);
(2) Fi Mary Vasil, Stanford University;
(3) Casidi Sujimoto, College of Public Policy, Georgia Institute of Technology;
(4) Haya Monroe White, Shahar College of Politics and Government and Computer Science Department, George Mason University ([email protected]).
Links table
Abstract and 1 introduction
1.1 Work and related contributions
2 methods and collection of data
2.1 Text identity agents and social and psychological damage
2.2 Sex modeling, sexual miles, and race
3 analysis
3.1 damage to omission
3.2 dependency damage
3.3 Step mold damage
4 discussion, thanks and appreciation, and references
Supplementary materials
Operating power and intersection
With extended technical details
B.1 Sex and sexual orientation
B.2 Modeling race
B.3 Automated data for text signals
B.4 Acting ratio
B.5 dependency ratio
B.
B.7 Steam Steam Analysis Step
B.8 Statistical methods
C additional examples
C.1 The most common names created by LM for each race
C.2 additional specific examples of complete artificial texts
D data data and general use disclosure
D.1 Data paper for Al -Shamszi, Data set claim
B.3 Automated data for text signals
To measure the damages of omission (see supplementary B.4), we collect 1000 generations for each language form for each directed to produce a sufficient number of total samples needed to model “N Small-n” groups. [35]. On the resulting data collection, which includes 500,000 stories, it is advisable to invest the text sermon from reading each individual story. Therefore, we are setting the GPT-3.5-Turbo to perform automatic references to the gender references and names with high accuracy.
First, we conclude the manually inferred sexes (based on the gender references) and the name on a set of evaluation of 4,600 generations is a unified story to the bottom of all five models, ensuring the representation of all three areas and both energy conditions equally. This then provides us with a set of sample data for accuracy statistics and summoning statistics on all 500,000 high confidence (.0063 95ci).
Next, we use ChatGPT 3.5 (GPT-3.5-Turbo) to make automated signs using the claim templates shown in the S7 table, chosen after repetition through the candidate claims and selection based on accuracy and recall. Depending on the scenarios and energy conditions for each of a specific story, see Appendix A, S3, S4, and S5 tables, we set the variable (variables) of the “personal” deputy in the rain template.
For each name we receive, then we try to analyze the JSON response that was returned to the performance of post -software treatment to remove hallucinations (such as references or names that are not in the text texts). We report the results of this initial process in the S8A table.
We note the results that fade with previous studies related to the joint reference decision that show automatic systems for weak identity groups [58]. For example, we note that the GPT-3.5-Turbo model that has been pre-trained does not work well for non-diodes like them, and often having difficulty distinguishing between decisions to individual characters against groups.
To address such problems, we do more permissible stories about the hands (outside the evaluation data set) with a particular focus on cases where we found the initial model of the conflict with non -dual pronouns in the field of love. This enhances our fatwa to the top of 98 % for both references and sexes, as shown in the S8B table. The final summons of the gender references reach 97 % for the gender references and above 99 % of the names.
We note that the formulation of a closed source model like Chatgpt has potential defects, including unconsciousness if the basic models change. In addition, at the time of writing this writing it had not released detailed information about the algorithms they used to control. For future work, the choice of the model is not restricted to Chatgpt, and OpenSource alternatives may also work.
B.4 Acting ratio
Using noticeable sweat and sex, we determine the amount of statistical proportions corresponding to the damages of omission and intervention. For the given demographic, we determine Acting ratio Proportion P Among the characters with a demographic monitored are divided into the proportion of the demographic designated in the comparison distribution P*.
The comparative distribution choice P* varies depending on the context required for the study. For example, it can be used to compare the percentage of the subject or profession (see S1 and S2 tables). Looking at pre -research, note how “fairness” definitions could block the regular challenges facing groups associated with the intersection [37]We instead focus on measuring the relative degree that our population composition is deleted in the study or excessively representing the social factors that already constitute the demographic composition to be unequal. Therefore, we set P* in our study to be the American census [83, 85]Noting that more progressive ideal for fairness (such as uniform groups (such as uniformly funded groups) cannot be achieved without exceeding the representation of the census (as a lower standard).
Six of seven racist categories are appointed as a number of 2022 [83]With the exception of the Middle East and North Africa, OB [57]. To calculate P* for sexual orientation and sexual identity (SOGI), we use home pulse scanning 2021 (HPS) [85]Studies have shown that they reduce the known issues of LGBTQ+ UnderCating [60]. See the S9 table to learn how to set SOGI to the sex type and relationship type chart.
