How to install latest Apache Spark on Ubuntu 16

In this Apache Spark Tutorial, we shall learn to install latest Apache Spark on Ubuntu 16.

Install dependencies first

Install Java

Open a terminal and run the following command to install Java :

sparkuser@tutorialkart:~$ sudo apt-get install default-jdk

Install latest Apache Spark on Ubuntu 16

Download Spark

Download latest Apache Spark release from http://spark.apache.org/downloads.html

Download and install latest Apache Spark on Ubuntu 16 - Apache Spark Tutorial - www.tutorialkart.com

Download Latest Apache Spark

Unzip and move spark to /usr/lib/

Open a terminal.

Unzip the downloaded .tgz file and move the folder to /usr/lib/ using the following commands :

sparkuser@tutorialkart:~$ tar xzvf spark-2.2.0-bin-hadoop2.7.tgz
sparkuser@tutorialkart:~$ mv spark-2.2.0-bin-hadoop2.7/ spark
sparkuser@tutorialkart:~$ sudo mv spark/ /usr/lib/

Add Path

Open ~/.bashrc with any of the editor and add Path to Java and Spark in ~/.bashrc . We shall use nano editor here :

sparkuser@tutorialkart:~$ sudo nano ~/.bashrc

Add following lines at the end of the file

export JAVA_HOME=/usr/lib/jvm/default-java/jre
export SPARK_HOME=/usr/lib/spark/bin
export PATH=$PATH:SPARK_HOME

Latest Apache Spark is successfully installed in your Ubuntu 16.

Verify installation

To verify the installation, close the Terminal already opened, and Open Terminal again. Run the following command :

sparkuser@tutorialkart:~$ spark-shell
Also verify the versions of Spark, Java and Scala displayed during the start of spark-shell.

:quit exits your from scala script of spark-shell.