Deep Learning with Swift for TensorFlow 📖

Official source code repository for "Deep Learning with Swift for TensorFlow" book.

🛍 @ Amazon (Paperback / Kindle) /  Books / Apress / SpringerLink

Book Details

About this book

Discover more insight about deep learning algorithms with Swift for TensorFlow. The Swift language was designed by Apple for optimized performance and development whereas TensorFlow library was designed by Google for advanced machine learning research. Swift for TensorFlow is a combination of both with support for modern hardware accelerators and more. This book covers the deep learning concepts from fundamentals to advanced research. It also introduces the Swift language for beginners in programming. This book is well suited for newcomers and experts in programming and deep learning alike. After reading this book you should be able to program various state-of-the-art deep learning algorithms yourself.

The book covers foundational concepts of machine learning. It also introduces the mathematics required to understand deep learning. Swift language is introduced such that it allows beginners and researchers to understand programming and easily transit to Swift for TensorFlow, respectively. You will understand the nuts and bolts of building and training neural networks, and build advanced algorithms.

What You’ll Learn

• Understand deep learning concepts
• Program various deep learning algorithms
• Run the algorithms in cloud

Who This Book Is For

• Newcomers to programming and/or deep learning, and experienced developers.
• Experienced deep learning practitioners and researchers who desire to work in user space instead of library space with a same programming language without compromising the speed.

Table of Contents

About the Author

About the Technical Reviewer

Preface

Chapter 1: Machine Learning Basics

 1.1 Machine Learning

  1.1.1 Experience

  1.1.2 Task

  1.1.3 Performance Measure

 1.2 Machine Learning Paradigms

  1.2.1 Supervised Learning

  1.2.2 Unsupervised Learning

  1.2.3 Semi-supervised Learning

  1.2.4 Reinforcement Learning

 1.3 Maximum Likelihood Estimation

 1.4 Elements of a Machine Learning Algorithm

  1.4.1 Data

  1.4.2 Models

  1.4.3 Loss Function

  1.4.4 Optimizer

  1.4.5 Regularizer

 1.5 Bias and Variance Trade-Off

 1.6 Why Deep Learning?

  1.6.1 Curse of Dimensionality

  1.6.2 Invalid Smoothness Assumption

  1.6.3 Deep Learning Advantages

 1.7 Summary

Chapter 2: Essential Math

 2.1 Linear Algebra

  2.1.1 Matrices and Vectors

  2.1.2 Unary Matrix Operations

  2.1.3 Binary Matrix Operations

  2.1.4 Norms

 2.2 Probability Theory

  2.2.1 Joint Probability

  2.2.2 Conditional Probability

  2.2.3 Elementary Rules

  2.2.4 Chain Rule

  2.2.5 Bayes Rule

 2.3 Differential Calculus

  2.3.1 Function

  2.3.2 Differentiation of Univariate Function

  2.3.3 Differentiation of Multivariate Function

  2.3.4 Differentiation of Vector Function

  2.3.5 Differentiation of Matrix Function

 2.4 Summary

Chapter 3: Differentiable Programming

 3.1 Swift is Everywhere

 3.2 Swift for TensorFlow

 3.3 Algorithmic Differentiation

  3.3.1 Programming Approaches

  3.3.2 Accumulation Modes

  3.3.3 Implementation Approaches

 3.4 Swift Language

  3.4.1 Values

  3.4.2 Collections

  3.4.3 Control Flow

  3.4.4 Closures and Functions

  3.4.5 Custom Types

  3.4.6 Modern Features

  3.4.7 Error Handling

  3.4.8 Advanced Operators

  3.4.9 Differentiation

 3.5 Python Interoperability

 3.6 Summary

Chapter 4: TensorFlow Basics

 4.1 Tensor

 4.2 Dataset Loading

  4.2.1 Epochs and Batches

 4.3 Defining Model

  4.3.1 Neural Network Protocols

  4.3.2 Sequence of Layers

 4.4 Training and Testing

  4.4.1 Checkpointing

  4.4.2 Model Optimization

  4.4.3 TrainingLoop

 4.5 From Scratch for Research

  4.5.1 Layer

  4.5.2 Activation Function

  4.5.3 Loss Function

  4.5.4 Optimizer

 4.6 Summary

Chapter 5: Neural Networks

 5.1 Gradient-Based Optimization

  5.1.1 Maxima, Minima, and Saddle Points

  5.1.2 Input Optimization

  5.1.3 Parameters Optimization

 5.2 Linear Models

  5.2.1 Regression

  5.2.2 Classification

 5.3 Deep Neural Network

  5.3.1 Dense Neural Network

 5.4 Activation Functions

  5.4.1 Sigmoid

  5.4.2 Softmax

  5.4.3 ReLU

  5.4.4 ELU

  5.4.5 Leaky ReLU

  5.4.6 SELU

 5.5 Loss Functions

  5.5.1 Sum of Squares

  5.5.2 Sigmoid Cross-Entropy

  5.5.3 Softmax Cross-Entropy

 5.6 Optimization

  5.6.1 Gradient Descent

  5.6.2 Momentum

 5.7 Regularization

  5.7.1 Dataset

  5.7.2 Architecture

  5.7.3 Loss Function

  5.7.4 Optimization

 5.8 Summary

Chapter 6: Computer Vision

 6.1 Convolutional Neural Network

  6.1.1 Convolution Layer

  6.1.2 Dimensions Calculation

  6.1.3 Pooling Layer

  6.1.4 Upsampling

 6.2 Prominent Features

  6.2.1 Local Connectivity

  6.2.2 Parameter Sharing

  6.2.3 Translation Equivariance

 6.3 Shortcut Connection

 6.4 Image Recognition

 6.5 Conclusion

References

Index

---

Setup

• First, you should install the latest Swift for TensorFlow toolchain.
• But to run only differentiation-specific code (for instance, the source code of Differentiable Programming chapter) not requiring deep learning features, simply install the latest Swift toolchain snapshot from Swift.org under Trunk Development (main) section. (The future differentiation feature updates will be posted on this website and will go through the standard Swift Evolution process.)
• Then you must select the newly installed toolchain in Xcode from Preferences (Command + ,) > Components > Toolchains > (Swift for TensorFlow or Swift Development Snapshot).

---

Executable Source Code

This Swift package offers various executable targets listed in Package.swift.

List of all executable targets:

DifferentiableProgramming

• AdvancedOperators
• AlgorithmicDifferentiation
• Arrays
• Classes
• Closures
• ConditionalStatements
• ControlTransfer
• Dictionaries
• Differentiation
• EarlyExit
• Enumerations
• ErrorHandling
• Extensions
• Generics
• GlobalFunctions
• Loops
• NestedFunctions
• Protocols
• PythonInteroperability
• Sets
• Structures
• Values

TensorFlowBasics

• EpochAndBatches
• FromScratchForResearch
• ModelDefinition
• TensorExplanation
• TrainingAndTesting
• TrainingLoopExample

NeuralNetworks

• InputOptimization
• LinearRegression
• ParametersOptimization
• PolynomialRegression

ComputerVision

• ImageRecognition

Running in Xcode

Select the executable target (for instance, AlgorithmicDifferentiation) from the scheme drop-down menu and then click the Run button. And result should be presented in the console.

Running in Terminal

To execute any of them first enter into this package's root directory.

cd deep-learning-with-swift-for-tensorflow-book

Then run the following command in terminal.

swift run AlgorithmicDifferentiation

The following output will be displayed.

expression value: 30.0
expression derivative: 28.0

Just replace AlgorithmicDifferentiation with any executable target you wish to run.

---

Note: This Swift package was tested on Xcode 12.5 running on macOS version 11.3. It should also work right out of the box on Linux distributions.

Contact

📫 [email protected]