Phi-3-MINI: PowerHouse 3.8B display LLM on your phone

Links table

Abstract and 1 introduction

2 technical specifications

3 academic standards

4 safety

5 times

6 Phi-3-Vision

6.1 Technical specifications

6.2 Academic standards

6.3 Safety

6.4 Weakness

Reference

Example example of criteria

Authors with (Abjdi)

C thanks and appreciation

a summary

We offer Phi-3-MINI, a 3.8 billion parametering language model that was trained on 3.3 trillion symbol, which was generally measured, as measured by both academic standards and internal tests, competitors models such as MMLU 8x7B and GPT-3.5 (EG, PHI-3-MINI achieve 69 % on MMLU. The entire innovation lies in our collection of data set For training, which is a copy of the version used in the phi-2, which consists of the generally available web data and artificial data. This model is also aligned with durability, safety and chat. Phi-3-MINI (for example, respectively 75 % and 78 % on MMLU, and 8.7 and 8.9 on MT-Bench). Moreover, we also offer Phi-Vision, a parameter model 4.2 billion dependent on Phi-3-MINI with strong thinking possibilities for image and text claims.

1 introduction

The amazing progress of Amnesty International in the past few years can be largely attributed to major efforts around the world towards expanding the scope of models and disturbing data groups. The size of the large LLMS models increased steadily from just one billion teachers only five years ago (GPT-2 was 1.5 billion teachers [RWC+ 19]) To trillion teachers today. The motivation of this effort arises in the apparently predictable improvement by training large models, so -called limitation laws [KMH+ 20, HBM+ 22, MRB+ 23]. However, these laws assume the source of “fixed” data. This assumption is now greatly disrupted by the presence of Llms Frontier, allowing us to interact with data in new ways. In our previous works on Phi models [GZA+ 23, LBE+ 23, JBA+ 23] It has been shown that a set of LLM liquidation of the web data available to the public, and the artificial data created from LLM, allows performance in the models of the smaller language that were usually seen in much larger models. For example, our previous model on this data recipe, Phi-2 (2.7B parameters), matching the performance of models that have been trained on regular data 25 times. In this report, we present a new model, Phi-3-MINI (3.8B parameters), trained on 3.3T features on larger and more advanced versions of data collections used in Phi-2. With its small size, the Phi-3-MINI can be easily concluded on a modern phone (see Figure 2), however it achieves a style quality with models like Mixtral 8x7b [JSR+ 24] And GPT-3.5.