Mesos: Difference between revisions

From Skytech
Jump to navigation Jump to search
(Created page with "Category:Linux = Installing mesos-master = <pre> ## Installing mesos ## apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF DISTRO=$(lsb_release -is | tr '[:uppe...")
 
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 69: Line 69:
== Spark ==
== Spark ==
Note: It is advisable to install hadoop or s3 first - either is required.
Note: It is advisable to install hadoop or s3 first - either is required.
http://spark.apache.org/docs/latest/running-on-mesos.html
* http://spark.apache.org/docs/latest/running-on-mesos.html


== Hadoop ==
== Hadoop ==
* https://github.com/mesos/hadoop ( Preferred method)
* https://strat0sphere.wordpress.com/2014/10/30/hadoop-on-mesos-installation-guide/ (2014)


= Mesos Links =
= Mesos Links =
Line 84: Line 86:
* https://www.digitalocean.com/community/tutorials/an-introduction-to-mesosphere
* https://www.digitalocean.com/community/tutorials/an-introduction-to-mesosphere
* https://www.digitalocean.com/community/tutorials/how-to-configure-a-production-ready-mesosphere-cluster-on-ubuntu-14-04
* https://www.digitalocean.com/community/tutorials/how-to-configure-a-production-ready-mesosphere-cluster-on-ubuntu-14-04

= Various links =
== Mesos vs Hadoop (YARN) ==
* https://www.mapr.com/blog/apache-mesos-vs-hadoop-yarn-whiteboard-walkthrough
* http://www.agildata.com/apache-spark-cluster-managers-yarn-mesos-or-standalone/
* https://www.quora.com/How-does-YARN-compare-to-Mesos

Latest revision as of 13:58, 29 June 2017


Installing mesos-master

## Installing mesos ##
apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
CODENAME=$(lsb_release -cs)
echo "deb http://repos.mesosphere.com/${DISTRO} ${CODENAME} main" | tee /etc/apt/sources.list.d/mesosphere.list

## Need java8-runtime-headless to run marathon (jessie-backports)
echo "deb http://ftp.debian.org/debian jessie-backports main" | tee  /etc/apt/sources.list.d/backports.list
apt-get update
apt-get -y install mesos marathon chronos
 
## Zookeeper
Make sure zooker has been setup and is in quorum. Create new zkpool:
Login to zookeeper ( /usr/share/zookeeper/bin/zkCli.sh ) and create:
create /mesos mesos
Created /mesos
## Edit mesos to speak to zookeeper:
$ cat /etc/mesos/zk
zk://192.168.0.61:2181,192.168.0.62:2181,192.168.0.63:2181/mesos
 
## Edit master setup to enable quorom:
Change quorum requirements from 1 -> 2
$ cat /etc/mesos-master/QUORUM
2

## Restart mesos-service
systemctl restart mesos-master.service

Installing mesos-slave nodes

## Installing mesos ##
apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
CODENAME=$(lsb_release -cs)
echo "deb http://repos.mesosphere.com/${DISTRO} ${CODENAME} main" | tee /etc/apt/sources.list.d/mesosphere.list
## Setup mesos-slave with ip for hostname+ip ##
echo "192.168.0.61" > cat /etc/mesos-slave/ip
cat ip > /etc/mesos-slave/hostname

## Edit mesos to speak to zookeeper
$ cat /etc/mesos/zk
zk://192.168.0.61:2181,192.168.0.62:2181,192.168.0.63:2181/mesos
 
## Restart mesos-service
systemctl restart mesos-slave.service

Install marathon

## Marathon ##
mkdir -p /etc/marathon/conf
cp /etc/mesos-master/hostname /etc/marathon/conf
cp /etc/mesos/zk /etc/marathon/conf/master
## Copy master (zk) config into a zk file and change endpoint to zk:/...../marathon
cp /etc/marathon/conf/master /etc/marathon/conf/zk 
 
## Choose jdk8 instead of 7 or marathon will fail and die horribly
update-alternatives --config java
 
systemctl restart marathon.service

Installing other frameworks

Spark

Note: It is advisable to install hadoop or s3 first - either is required.

Hadoop

Mesos Links

Autoscaling mesos applications / frameworks

Howto

Various links

Mesos vs Hadoop (YARN)