Links table
Abstract and 1. Introduction
-
Related work
-
methodology
-
Experimental preparation and results
-
Conclusion and future work
Thanks and appreciation
Reproduction statement
Exponation statement and references
4. Experimental preparation and results
We expand the experimental standard that Wu and others have made. (2023) through different dimensions. Below, we summarize the design options for our standard and highlight the main differences of the time network[5].
Modeling the time chain with limited supervision. Our indicator consists of 5 tasks modeling from the main temporal chain of an important practical value, which is a long prediction, abbreviation, verification, classification, and detection of anomalies, as shown in the tab. 1. Unlike the time network, we look exclusively in the scenarios that feature limited account resources and supervision. These scenarios mimic practical situations in which training (or accurate control) is a deep nerve network that is not possible due to the restrictions of resources or data that is not adequately distinguished. Accordingly, we evaluate the moment in zero sets whenever possible and through the linear investigation of a few times otherwise.
As for the classification, we consider the problem of learning to be not subject to supervision, as the goal is to learn the representations of the useful chains that are useful for classifying the estuary, without accessing the named data. As is common in previous work (Yue et al., 2022; Franceschi et al To predict the short horizon, we consider the zero preparation that Oreshkin et al. (2021). In particular, we define a moment to the source data set using the forecasting head,
Its performance is evaluated on a targeted data set without any refinement (App e.1.2, tab 21).
Data groups. We use the same data groups as intervention to predict and collect. However, for classification and abnormal detection, we conduct experiments on a larger sub -group and systematically choose from data collections from UCR classification (Dau Et Al Specifically, we run classification experiences on all 91 time chain data sets with each short time chain than 512 time steps (TAB.23). To discover anomalies, while choosing a sub -set of time chain, we have prepared a priority to cover various fields and data sources represented in the UCR an archive (tab 22). We also note that the UCR ancient archive was suggested as an improvement in the pre -existing anomalies detection collections such as SMD (SU ET Al The proposed experimental setting was summarized on the tab. 1 and detail in the application. E.
Scales. We evaluate each experience using multiple standards used in the task standards, such as MSE and MAE for long -horizontal prediction, and SMAPE to predict the short horizon. We also note that temporal fabric and GPT4ts (Zhou et al Instead, we measure the performance of anomalies with the best modified F1 degree used widely (Goswami et al., 2023A; chalu et al
Foundation. We compared a moment with deep learning models and statistical learning through tasks (tab 35). This is unlike the time network that mainly compares to the transformer -based methods. These comparisons are necessary to assess the practical benefit of the proposed methods. We have found that statistical approaches and non-transfers such as ARIMA to predict ShortORIZON, N-Beats to predict the long horizon, and K-Nearest neighbors to detect anomalies outperforming many deep and transformed models.
Set the parameter Hyper. We do not control the super. In all the experiments that follow, unless it is mentioned otherwise, we bear a moment with the size of a batch of 64, and the schedule of one cycle learning rate with the peak learning rate between 5E-5 and 1e-3 (Smith & Topin, 2019). For baseline methods, we pick up the recommended settings from their papers and their general warehouses. We report all excessive parameters settings for a moment and basic lines in the application. E.
Search questions. Through the following experiences, we aim to answer 3 wide research questions.
RQ1: Effectiveness. Is an effective moment for the tasks of analyzing the time chains multiple in limited supervision settings?
RQ2: Interpretation. What is learning for a moment? Do you pick up the characteristics of intuitive time chains such as frequencies, trends and changing capacity?
RQ3: characteristics. What is the effect of the size of the size model? Is it possible to use a moment, closer to LLMS, to learn cross -transport?
4.1. The moment of multiple time chains modeling tasks can be resolved in limited supervision settings
Long horizon prediction. The moment of investigation is achieved in line with the latest performance on most data and prospects, and takes only second place in the correction that generally achieves the lowest MSE (tab 2). In many databases and prospects, prediction models on the basis of LLMS – Timellm and GPT4TS are worse than the moment. It is worth noting that Nbeats outperforms many modern methods, with a focus on the importance of comparing the prediction performance that exceeds transformer -based methods.
Zero Shoot Short Horizon predict. Of all the tasks, we found that the prediction of a sacraver is under the largest scope of improvement (tab 3). Statistical methods such as Theta and ETS outperformed their deepest counterparts. However, in some data sets, the SMAPE moment has achieved less than ARIMA.
classification. Without any special refinement of the data, the moment of learning distinctive representations of different categories of data (Figure 5) can be learned, and SVM is better on its representations in all ways except for 4 styles specifically designed for temporal chain classification models and training them on each individual data group. The recently proposed GPT4TS performance and poorly clarifying the times despite training on each individual data group with stickers.
Discovering anomalies. In 44 time series of UCR an archive, the moment has been constantly outperformed on both time and GPT4TS, as well as deep educational models of the category designed to discover anomalies, in both zero and written investigation formations. However, the performance of the K-Nearest neighbors was marginally better in terms of VUS-ROC, but they had the best modified F1 degree.
Inclusion. Unpaid bill. 6 contains the performance of all models, average of more than 4 different hiding rates. The moment with the linear investigation made the slightest error in reconstruction on all ETT data sets. In the preparation of zero, the moment continuously outperformed all the statistical fulfillment methods except for the written fulfillment.
4.2. What is learning for a moment?
We have found that the moment can capture changes in the characteristics of intuitive time chains such as trend, capacity, frequencies and stages of time chains. However, it is not possible to distinguish between the timeline that was transmitted vertically because it normalizes each signal before modeling (Figure 4,7). Moreover, in many classification data collections, the moment learns distinctive representations for different categories, even in a zero setting without reaching stickers (Figure 5, 8).
4.3. The characteristics of large time chains models
The scaling model improves training loss. Like LLMS, we found that increasing the size of the model leads to low training loss, even before the first era (Figure 6, left). The next immediate step is to assess the effectiveness of this phenomenon to the tasks of modeling time chains under limited supervision.
The moment can be solved by series learning tasks via media. If and others. (2022) First shows that the large language transformers that were trained before training can solve the general serial learning tasks of the methods outside the text and minimal images. Several recent studies have benefited from these characteristics to reprogram LLMS for the tasks of time chains. We explore whether it is also possible to use the pre -trained transformers on the time series to solve the tasks of rating on data, text and bilateral data. Our results confirm that by freezing self-feeding layers, a moment can be designed similar to GPT-2 and Flan-T5 models with a similar range (tab 5).
A moment with the random preparation weights is close to the low training loss. Our notes indicate that with adequate data, our pre -zero training leads to a decrease in the loss of training from continuous training on a similar model that has been prepared with language modeling weights (Figure 6, 12). This also confirms that there are pre -training data that can be accessed in general available in the time chain pile to facilitate the basis forms of the time series before training from the zero point.
Authors:
(1) Mononito Goswami, Auton Lab, Robotics Insitute, Carnegie Mellon University, Pittsburgh, USA ([email protected]))
(2) Konrad Szafer, Auton Lab, Robotat Institute, Carnegie Mellon University, Pittsburgh, USA, with an equal contribution, decided to request the use of a random generator;
(3) Arjun Chaudhry, Auton Laboratory, Robotat Institute, University of Carnegie Mellon, Pittsburg, USA, with an equal contribution, decided to use a random generator;
(4) Yifu Cai, Auton Lab, Robotat Institute, Carnegie Mellon University, Pittsburgh, USA;
(5) Shu Lee, University of Pennsylvania, Philadelphia, USA;
(6) Artur Dubrawski, Auton LAB, Android Institute, Carnegie Mellon University, Pittsburg, USA.
[5] In this section, we use the time network to refer to the standard suggested by Wu and others. (2023) Instead of their model.
