Sunday, 19 June 2016


Apache Spark

               Downloading and Running Spark on windows

Spark is 3rd generation distributed technologies,

Download build using below URL.

1) www.spark.apache.org -> Choose Spark-build with pre-build for hadoop 2.x version
2) Extract Download and bin folder. 
3) Open prompt and navigate Saprk/bin location.
4) Type Spark-Shell and Opens Scala> prompt
5) Open Spark Admin UI and type below URL

http://localhost:4040 ( Spark Web UI)

Sample Program to Test Spark Environment:
> val  v1 = 1 to 100
> val dv1 = sc.parallelize(v1)
> dv1.filter(_<10).collect()
> val result = dv1.filter(_<10).collect()
> result

Output: array of numbers ( 0, 1, 2, 3, 4, 5, 6, 7, 8, 9)





No comments:

Post a Comment