10 Gradient Descent Optimisation Algorithms

Gradient descent is an optimisation method for finding the minimum of a function.

It is commonly used in deep learning models to update the weights of the neural network through backpropagation.

In this article, we will summarise the common gradient descent optimisation algorithms used in popular deep learning frameworks (e.g. TensorFlow, Keras, PyTorch, Caffe). The purpose of this post is to make it easy to read and digest (using consistent nomenclature) since there aren’t many of such summaries out there, and as a cheat sheet if you want to implement them from scratch.


skeleton of using logistic regression in R.


#load data
train <- read.csv(‘Train_Old.csv’)

#create training and validation data from given data

split <- sample.split(train$Recommended, SplitRatio = 0.75)

#get training and test data
dresstrain <- subset(train, split == TRUE)
dresstest <- subset(train, split == FALSE)

#logistic regression model
model <- glm (Recommended ~ .-ID, data = dresstrain, family = binomial)

predict <- predict(model, type = ‘response’)

#confusion matrix
table(dresstrain$Recommended, predict > 0.5)

#ROCR Curve
ROCRpred <- prediction(predict, dresstrain$Recommended)
ROCRperf <- performance(ROCRpred, ‘tpr’,’fpr’)
plot(ROCRperf, colorize = TRUE, text.adj = c(-0.2,1.7))

#plot glm
ggplot(dresstrain, aes(x=Rating, y=Recommended)) + geom_point() +
stat_smooth(method=”glm”, family=”binomial”, se=FALSE)

opencv news

OpenCV (Open Source Computer Vision Library)

is released under a BSD license and hence it’s free for both academic and commercial use. It has C++, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android. OpenCV was designed for computational efficiency and with a strong focus on real-time applications.

Written in optimized C/C++, the library can take advantage of multi-core processing. Enabled with OpenCL, it can take advantage of the hardware acceleration of the underlying heterogeneous compute platform.

Usage ranges from interactive art, to mines inspection, stitching maps on the web or through advanced robotics.

There are many different ways to do image recognition.
Google recently released a new Tensorflow Object Detection API to give computer vision everywhere a boost
Tensorflow Object Detection API is a very powerful source for quickly building object detection models.

Understanding the API

The API has been trained on the COCO dataset (Common Objects in Context).
This is a dataset of 300k images of 90 most commonly found objects.
Examples of objects includes:


Openstack in a good cloud managed infrastructure

OpenStack is a cloud computing system written in Python.

The project is 8 years old.

During that time we have had 18 major releases, and to give you an idea of the size of the community, during 2017, we had 2,500 people contribute more than 65,000 patches. The age and size of the project means our community has been through several transitions and challenges that other smaller projects may not yet have encountered, and my hope is that by thinking about them early, you can prepare for them, and your communities will be able to grow in a healthier way.

In 2019,
A good cloud managed infrastructure has grown by a factor of 10 compared to the resources in 2013.
This has been achieved in collaboration with the many open source communities we have worked with over the past years,
including :


What’s new in 4.2.0 ?

  • Fix a bug with empty word lists (contributed by FabioRosado)
  • Update dependency management to use setuptools extras Document how to create multiple wordfiles (contributed by FabioRosado)
  • Note that PyEnchant is unmaintained and fix links (contributed by Marti Raudsepp)
  • Don’t use mutable default argument (contributed by Daniele Tricoli)