Big Data Companies In Nepal


Companies

Use cases

Using Hadoop/Spark

FusemachinesSales Lead GenerationSpark
Deerwalk U.S. health care industryHadoop
Verscend or Versand Technologies Pvt. Ltd.U.S. health care industryHadoop or Cloudera
CloudFactoryhttps://www.cloudfactory.com/use-cases Hadoop
JavraBig Data and BI, Progress ServicesHadoop
GrowByData SaaS development and Big Data analytics,Hadoop
GrepsrData enrichment Hadoop
Other companies are also trying to adapt
Engineering colleges have BIg data subjects
We are plannig to start Training centers on Big data

Comments

Post a Comment

Popular posts from this blog

DIfferent issues that may occur in Apache spark and their remedies.

DIfferent issues that may occur in spark and their remedies : Currently we are using m4.2xlarge Spark jobs are performed efficiently to process the large data with the configurations discussed below,  taming the big data to get desired output with low latency.    Here we discuss the Spark configuration parameters we applied to resolve issues ,and get efficient performance in AWS to process Big data of 30 gb. Spark on yarn environment: (set, below two properties to submit job though spark-submit. –num-executors NUM     Number of executors to launch (Default: 2). –executor-cores NUM     Number of cores per executor (Default: 1). Note: These switches to be used depending upon cluster capacity. Troubleshooting: Issue 1: Exception in thread “main” org.apache.spark.SparkException: A master URL must be configured. Resolution:      Spark properties are configured in three ways: Setting the configuration properties in the code using spark conf Setting
Read more

Parquet is a column based data store or File Format (Useful for Spark read/write and SQL in order to boost performance)

Image
                                                                 PARQUET FILE SYSTEM Parquet is a column based store. Parquet was a joint project of cloudera and Twitter engineers. It is built to support very efficient compression and encoding schemes. Parquet allows compression schemes to be specified on a per-column level, and is future-proofed to allow adding more encodings as they are invented and implemented. We separate the concepts of encoding and compression, allowing Parquet consumers to implement operators that work directly on encoded data without paying decompression and decoding penalty when possible. Twitter is starting to convert some of its major data source to Parquet in order to take advantage of the compression and deserialization savings. Fusemachines is AI based sales based company we used to use JSON as a data store. Now we are upgrading to Parquet data store file system.   Time and space analysis on Small Datasets (Statistics on
Read more

Steps to Install zeppelin with spark

Image
- Data Ingestion, Data Discovery,Data Analytics, Data Visualization &Collaboration we can use zeppelin, below are the steps to install it : 1. Download spark-2.1.0 2.Download zeppein-file from official site 3.Sudo vi .bashrc  , and place it in last export JAVA_HOME=/usr/lib/jvm/java-8-oracle export Spark_HOME=/home/ashis/Downloads/spark-2.1.0 5. Source .bashrc 6. cd Downloads/conf 5. Sudo vi zeppelin-env.sh Place the same code here , export JAVA_HOME=/usr/lib/jvm/java-8-oracle export Spark_HOME=/home/ashis/Downloads/spark-2.1.0 6.  Sudo vi zeppelin-site.xml Change the port to 8082 from 8080 7. Cd spark-2.1.0  sbin/start-all.sh 8.  Cd zeppelin     9. bin/zeppelin-daemon.sh  start 10.bin/spark-shell 11. http://localhost:8082/  for zeppelin UI 12. Try Zeppelin tutorial given here : http://localhost:8082/#/ or if its not working you can follow below link https://gist.github.com/pratos/b2e2937106980a867d0558cba
Read more