gtag('config', 'G-0PFHD683JR');
Price Prediction

How to train the linear slope in Tensorflow

Content overview

  • Proven
  • Solving machine learning problems
  • Data
  • Determine the form
  • Determine the loss function
  • Determine the training episode
  • The same solution, but with keras
  • The following steps

In the previous evidence, I learned about the conversations, variables, gradient tape and stereotypes. In this guide, you will suit all of this together to train models.

Tensorflow also includes the TF.Keras applications interface, which is a high -level nervous applications interface that provides useful abstraction to reduce the kettle. However, in this guide, basic seasons will be used.

Proven

import tensorflow as tf

import matplotlib.pyplot as plt

colors = plt.rcParams['axes.prop_cycle'].by_key()['color']

Solving machine learning problems

The solution to the problem of machine learning usually consists of the following steps:

  • Get training data.
  • Determine the form.
  • Determine the loss function.
  • Run training data, and calculate the loss of the ideal value
  • Calculate gradients for that loss and use Improved To adjust the variables to suit the data.
  • Evaluate your results.

For clarification purposes, in this guide, you will develop a simple linear model, F (x) = x ∗ w+b, which contains two variables: W (weights) and B (bias).

This is the simplest machine learning problems: looking at X and Y, try to find a slope and remove a line through a simple linear slope.

Data

Uses of learning subject to supervision Inputs (It usually indicates this x) And Outputs (Referred to it YOften called Labels). The goal is to learn from the associated inputs and outputs so that you can predict the value of the output from the inputs.

All inputs of your data are represented in Tensorflow, almost almost by tensioner, often vector. In supervision training, the output (or the value you want to predict) is also tense.

Below are some of the data that was manufactured by adding the high (natural) noise to points along the line.

# The actual line
TRUE_W = 3.0
TRUE_B = 2.0

NUM_EXAMPLES = 201

# A vector of random x values
x = tf.linspace(-2,2, NUM_EXAMPLES)
x = tf.cast(x, tf.float32)

def f(x):
  return x * TRUE_W + TRUE_B

# Generate some noise
noise = tf.random.normal(shape=[NUM_EXAMPLES])

# Calculate y
y = f(x) + noise
# Plot all the data
plt.plot(x, y, '.')
plt.show()

Repeating is usually collected together in BatchesOr sets of inputs and outputs are stacked together. The cavity can give some training advantages and work well with the accelerator and the directed account. Looking at how small this data collection, you can handle the entire data set as one batch.

Determine the form

Use tf.Variable To represent all weights in a model. A tf.Variable Store value and this provides a tensioner as needed. See the variable guide for more details.

Use tf.Module To package variables and account. You can use any Python object, but in this way it can be easily saved.

Here, both are determined W and for As variables.

class MyModel(tf.Module):
  def __init__(self, **kwargs):
    super().__init__(**kwargs)
    # Initialize the weights to `5.0` and the bias to `0.0`
    # In practice, these should be randomly initialized
    self.w = tf.Variable(5.0)
    self.b = tf.Variable(0.0)

  def __call__(self, x):
    return self.w * x + self.b

model = MyModel()

# List the variables tf.modules's built-in variable aggregation.
print("Variables:", model.variables)

# Verify the model works
assert model(3.0).numpy() == 15.0

The initial variables here are set in a fixed way, but KERAS comes with any of the number of gates that you can use, with or without the rest of the Keras.

Determine the loss function

The loss function measures the quality of the form of the model for a specific input that matches the target output. The goal is to reduce this difference during training. Determine the standard L2 loss, also known as the “medium Spring” error:

# This computes a single loss value for an entire batch
def loss(target_y, predicted_y):
  return tf.reduce_mean(tf.square(target_y - predicted_y))

Before training the form, you can imagine the value of the loss by drawing the form of the form in red and blue training data:

plt.plot(x, y, '.', label="Data")
plt.plot(x, f(x), label="Ground truth")
plt.plot(x, model(x), label="Predictions")
plt.legend()
plt.show()

print("Current loss: %1.6f" % loss(y, model(x)).numpy())

Determine the training episode

The training episode consists of three tasks over and over again:

  • Send a set of inputs through the form to create outputs
  • Calculate the loss by comparing the outputs with the output (or naming)
  • Using the gradient tape to find gradients
  • Improving variables with these gradients

In this example, you can train the model using the gradient.

There are many variables from the gradual descent plan that is captured in tf.keras.optimizers. But in the construction spirit of the first principles, you will implement basic mathematics on your own with the help of tf.GradientTape For automatic differentiation and tf.assign_sub To reduce the value (that combines tf.assign and tf.sub):

# Given a callable model, inputs, outputs, and a learning rate...
def train(model, x, y, learning_rate):

  with tf.GradientTape() as t:
    # Trainable variables are automatically tracked by GradientTape
    current_loss = loss(y, model(x))

  # Use GradientTape to calculate the gradients with respect to W and b
  dw, db = t.gradient(current_loss, [model.w, model.b])

  # Subtract the gradient scaled by the learning rate
  model.w.assign_sub(learning_rate * dw)
  model.b.assign_sub(learning_rate * db)

To look at the training, you can send the same batch x and Y Through the training episode, see how W and b development.

model = MyModel()

# Collect the history of W-values and b-values to plot later
weights = []
biases = []
epochs = range(10)

# Define a training loop
def report(model, loss):
  return f"W = {model.w.numpy():1.2f}, b = {model.b.numpy():1.2f}, loss={loss:2.5f}"


def training_loop(model, x, y):

  for epoch in epochs:
    # Update the model with the single giant batch
    train(model, x, y, learning_rate=0.1)

    # Track this before I update
    weights.append(model.w.numpy())
    biases.append(model.b.numpy())
    current_loss = loss(y, model(x))

    print(f"Epoch {epoch:2d}:")
    print("    ", report(model, current_loss))

Is training

current_loss = loss(y, model(x))

print(f"Starting:")
print("    ", report(model, current_loss))

training_loop(model, x, y)

Draw the development of weights over time:

plt.plot(epochs, weights, label='Weights', color=colors[0])
plt.plot(epochs, [TRUE_W] * len(epochs), '--',
         label = "True weight", color=colors[0])

plt.plot(epochs, biases, label='bias', color=colors[1])
plt.plot(epochs, [TRUE_B] * len(epochs), "--",
         label="True bias", color=colors[1])

plt.legend()
plt.show()

Imagine how the trained model performs

plt.plot(x, y, '.', label="Data")
plt.plot(x, f(x), label="Ground truth")
plt.plot(x, model(x), label="Predictions")
plt.legend()
plt.show()

print("Current loss: %1.6f" % loss(model(x), y).numpy())

The same solution, but with keras

It is useful to discuss the code above with its equivalent in Keras.

It seems that determining the model is completely the same if you are a sub -category tf.keras.Model. Remember that the Keras models inherit the end of the stereotype.

class MyModelKeras(tf.keras.Model):
  def __init__(self, **kwargs):
    super().__init__(**kwargs)
    # Initialize the weights to `5.0` and the bias to `0.0`
    # In practice, these should be randomly initialized
    self.w = tf.Variable(5.0)
    self.b = tf.Variable(0.0)

  def call(self, x):
    return self.w * x + self.b

keras_model = MyModelKeras()

# Reuse the training loop with a Keras model
training_loop(keras_model, x, y)

# You can also save a checkpoint using Keras's built-in support
keras_model.save_weights("my_checkpoint")

Instead of writing new training episodes every time you create a model, you can use the features of Keras as a shortcut. This can be useful when you don’t want to write or correct Python training episodes.

If you do, you will need to use model.compile() To set the teachers, and model.fit() For training. It can be a lower symbol of using KERAS applications for L2 loss and gradient rates, again as a shortcut. KERAS losses and their allicity can also be used outside these rest functions, and the previous example could have been used.

keras_model = MyModelKeras()

# compile sets the training parameters
keras_model.compile(
    # By default, fit() uses tf.function().  You can
    # turn that off for debugging, but it is on now.
    run_eagerly=False,

    # Using a built-in optimizer, configuring as an object
    optimizer=tf.keras.optimizers.SGD(learning_rate=0.1),

    # Keras comes with built-in MSE error
    # However, you could use the loss function
    # defined above
    loss=tf.keras.losses.mean_squared_error,
)

Kiras fit Double data or full data set as a Numby. Numby matches are cut into batches and failed to size 32.

In this case, to match the behavior of the episode written by hand, you must pass x In one go of 1000 size.

print(x.shape[0])
keras_model.fit(x, y, epochs=10, batch_size=1000)

Note that Keras prints the loss after training, not before, so the first loss appears less, but this mainly shows the performance of the training itself.

The following steps

In this guide, I saw how to use the basic layers of mintments, variables, stereotypes and gradients to build and train a model, and enhance how to map these ideas to Keras.

This, however, is a very simple problem. For a more practical introduction, see the allocated training tour.

To learn more about the use of integrated Keras training rings, see this guide. To learn more about training seminars and karas, see this guide. To write allocated distributed training episodes, see this guide.


It was originally published on Tensorflow Web site, this article appears here under a new and licensed title under CC by 4.0. Common icon samples under APache 2.0 license.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button