bit.ly/pybay-keras

Who Am I?

Brian Spiering

What Do I Do?

Professor @

Keras - Neural Networks for humans

A high-level, intuitive API for Deep Learning.

Easy to define neural networks, then automatically handles execution.

A simple, modular interface which allows focus on learning and enables fast experimentation

Goals

• General introduction to Deep Learning
• Overview of keras library
• An end-to-end example in keras

Anti-Goals

• Understanding of Deep Learning (there will be no equations)
• Building neural networks from scratch
• Complete survey of keras library

Deep Learning 101

Deep Learning (DL) are Neural networks (NN) with >1 hidden layer

Neural Networks are Nodes & Edges

Nonlinear function allows learning of nonlinear relationships

Groups of nodes all the way down

Deep Learning isn't magic, it is just very good at finding patterns

Deep Learning has fewer steps than traditional Machine Learning

If you want to follow along…

GitHub repo: bit.ly/pybay-keras

If you want to type along…

1. Run a local Jupyter Notebook
2. Binder: In-Browser Jupyter Notebook
3. Colaboratory: "Google Docs for Jupyter Notebooks"

import keras

# What is the backend / execution engine?

"An open-source software library for Machine Intelligence"

Numerical computation using data flow graphs.

TensorFlow: A great backend

A very flexible architecture which allows you to do almost any numerical operation.

Then deploy the computation to CPUs or GPUs (one or more) across desktop, cloud, or mobile device.

MNIST handwritten digit database:
The “Hello World!” of Computer Vision

# Import data

# Setup train and test splits

from random import randint
from matplotlib import pyplot

%matplotlib inline

pyplot.imshow(x_train[randint(0, x_train.shape[0])], cmap='gray_r');

# Munge Data
# Transform from matrix to vector, cast, and normalize

# Convert class vectors to binary class matrices

# Import the most common type of neural network

# Define model instance

# Import the most common type of network layer, fully interconnected

# Define input layer

# Define another layer

# Define output layers

# Print summary

# Yes - we compile the model to run it

# Train the model

# Let's see how well our model performs

Keras' Other Features

• Common built-in functions (e.g., activation functions and optimitizers)
• Convolutional neural network (CNN or ConvNet)
• Recurrent neural network (RNN) & Long-short term memory (LSTM)
• Pre-trained models

Summary

• Keras is designed for human beings, not computers.
• Easier to try out Deep Learning (focus on the what, not the how).
• Simple to define neural networks.

Futher Study - Keras

• Keras docs
• Keras blog
• Keras courses
  • edX
  • Coursera

Futher Study - Deep Learning

• Prerequisites: Linear Algebra, Probability, Machine Learning
• fast.ai Course
• Deep Learning Book

Bonus Material

# reset -fs

# from keras import *

# whos

# from keras.datasets import fashion_mnist

# # Setup train and test splits
# (x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

# from random import randint
# from matplotlib import pyplot

# %matplotlib inline

# pyplot.imshow(x_train[randint(0, x_train.shape[0])], cmap='gray_r');

# # Define CNN model

# # Redefine input dimensions to make sure conv works
# img_rows, img_cols = 28, 28
# x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
# x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
# input_shape = (img_rows, img_cols, 1)

# import keras

# # Convert class vectors to binary class matrices
# y_train = keras.utils.to_categorical(y_train, 10)
# y_test = keras.utils.to_categorical(y_test, 10)

# from keras.layers import Conv2D, Dense, Flatten, MaxPooling2D

# # Define model
# model = Sequential()
# model.add(Conv2D(32, 
#              kernel_size=(3, 3),
#              activation='sigmoid',
#              input_shape=input_shape))
# model.add(Conv2D(64, (3, 3), activation='sigmoid'))
# model.add(MaxPooling2D(pool_size=(2, 2)))
# model.add(Flatten())
# model.add(Dense(128, activation='sigmoid'))
# model.add(Dense(10, activation='softmax'))

# model.compile(loss='categorical_crossentropy', 
#               optimizer='adam',
#               metrics=['accuracy'])

# # Define training
# training = model.fit(x_train, 
#                      y_train,
#                      epochs=5,
#                      verbose=True, 
#                      validation_split=0.1)

# loss, accuracy = model.evaluate(x_test, 
#                                 y_test, 
#                                 verbose=True)
# print(f"Test loss: {loss:.3}")
# print(f"Test accuracy: {accuracy:.3%}")

What is keras?

Keras (κέρας) means horn in Greek.

It is a reference to a literary image from ancient Greek and Latin literature.

First found in the Odyssey, where dream spirits (Oneiroi, singular Oneiros) are divided between those who deceive men with false visions, who arrive to Earth through a gate of ivory, and those who announce a future that will come to pass, who arrive through a gate of horn.

It's a play on the words κέρας (horn) / κραίνω (fulfill), and ἐλέφας (ivory) / ἐλεφαίρομαι (deceive).