Docker小试牛刀，集群部署简单WEB应用

by

2018-04-26

01:54:00

Leave a comment on Docker小试牛刀，集群部署简单WEB应用

Docker/Kubernetes

要做的一件事，通过Docker部署一套WEB服务，用户进行访问HA节点进行负载均衡，进行分发调度到后端两个Server节点，一个Master数据库节点和两个Slave数据库节点，当然这里说的节点最终都部署为Docker容器，中间遇到了一些问题，记录一下

部署环境如下：

主机：VMWare Fusion虚拟机

操作系统：Ubuntu 16.04.4 LTS

内核版本：4.4.0-116-generic

Docker版本：Docker version 18.04.0-ce, build 3d479c0

大致结构图如下

NewImage

 

 

 

 

 

 

比较普通直接的流程，对外用户暴露的就是HA的IP地址和端口，这样用户通过访问HA，就会调度到后端WEB Server然后进行相应请求

先从Docker Hub里拉相关的镜像，具体包括

lihui@docker:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
haproxy             latest              2b6a4216580b        5 days ago          74.4MB
ubuntu              latest              c9d990395902        12 days ago         113MB
redis               latest              c5355f8853e4        4 weeks ago         107MB
django              latest              eb40dcf64078        16 months ago       436MB

容器启动顺序：redis-master => redis-slave => app => ha 

具体如下：

  1. 启动redis-master容器
  2. 两个redis-slave容器启动时连接到redis-master上
  3. 两个app容器启动时连接到redis-master上
  4. ha容器启动时连接到两个app上

启动容器

首先启动redis-master容器

lihui@docker:~$ sudo docker run -it --name redis-master redis /bin/bash
root@2c7d734df8d4:/data#

然后启动两个redis-slave容器

redis-slave1

lihui@docker:~$ sudo docker run -it --name redis-slave1 --link redis-master:master redis /bin/bash
root@4b03b99660fb:/data#

redis-slave2

lihui@docker:~$ sudo docker run -it --name redis-slave2 --link redis-master:master redis /bin/bash
root@c16515555d20:/data#

这里可以看到slave的启动和master稍有差别，多了一个–link选项，可以通过容器名来确定对应的容器

比如redis-slave1容器里，除了正常172.17.0.3和hostname的对应关系外，还记录了名称为master的连接信息，redis-master容器对应的IP地址为172.17.0.2

lihui@docker:~$ sudo docker run -it --name redis-slave1 --link redis-master:master redis /bin/bash
root@4b03b99660fb:/data# cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2	master 2c7d734df8d4 redis-master
172.17.0.3	4b03b99660fb

而redis-master容器里，却没有该类似信息

lihui@docker:~$ sudo docker run -it --name redis-master redis /bin/bash
root@2c7d734df8d4:/data# cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2	2c7d734df8d4

接着启动app容器，这里就是Django容器

APP1

lihui@docker:~$ sudo docker run -it --name APP1 --link redis-master:db -v ~/Projects/Django/App1:/usr/src/app django /bin/bash
root@e3f4df84474c:/#

APP2

lihui@docker:~$ sudo docker run -it --name APP2 --link redis-master:db -v ~/Projects/Django/App2:/usr/src/app django /bin/bash
root@81382a640c1a:/#

最后启动ha容器，端口挑一个没用到的

HAProxy

lihui@docker:~$ sudo docker run -it --name HAProxy --link APP1:APP1 --link APP2:APP2 -p 4444:4444 -v ~/Projects/HAProxy:/tmp haproxy /bin/bash
root@f18e99c6ddef:/#

这样，相关的容器都已经启动完毕了，查看一下

lihui@docker:~$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                    NAMES
f18e99c6ddef        haproxy             "/docker-entrypoint.…"   About a minute ago   Up About a minute   0.0.0.0:4444->4444/tcp   HAProxy
81382a640c1a        django              "/bin/bash"              7 minutes ago        Up 7 minutes                                 APP2
e3f4df84474c        django              "/bin/bash"              9 minutes ago        Up 9 minutes                                 APP1
c16515555d20        redis               "docker-entrypoint.s…"   20 minutes ago       Up 20 minutes       6379/tcp                 redis-slave2
4b03b99660fb        redis               "docker-entrypoint.s…"   21 minutes ago       Up 21 minutes       6379/tcp                 redis-slave1
2c7d734df8d4        redis               "docker-entrypoint.s…"   23 minutes ago       Up 23 minutes       6379/tcp                 redis-master

查看veth pair的个数，也是6个，和容器数一致

lihui@docker:~$ ip l | grep veth | wc -l
6

更新配置

当前只是启动了容器，而如果要正确地启动每个服务，需要根据自己的配置进行更新，才能完成通信以及相应的功能

由于容器本身就是轻量级，不可能像一个KVM一个Linux ISO那样，携带一些常用工具，正常连文本编辑器也没有，当然如果你想联网安装对应的软件包或者上传安装包也是可以的，但是其实有种更简单的方法，通过volume来实现创建文件；在容器启动时，指定-v参数挂载volume，，这样主机和容器之间就可以共享数据，就可以直接主机上进行操作；上面启动容器的时候可以看到redis并没有指定volume，原因是redis镜像里已经集成了volume挂载命令，因此在启动后就已经挂载好了，直接用即可

redis-master容器配置

先在host节点上查看一下挂载的volume在哪里

lihui@docker:~$ sudo docker inspect redis-master | grep -i mounts -A 10
        "Mounts": [
            {
                "Type": "volume",
                "Name": "23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0",
                "Source": "/var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }

可见主机目录为/var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data，对应着容器的目录为/data

可以先做一下简单的测试，主机该目录写一个文件

lihui@docker:~$ sudo touch /var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data/hi
lihui@docker:~$ sudo ls -l /var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data/
total 0
-rw-r--r-- 1 root root 0 Apr 25 20:13 hi

查看容器里面

root@2c7d734df8d4:/data# ls -l
total 0
-rw-r--r-- 1 root root 0 Apr 26 03:13 hi

这里时区没对上，先不用管，可见可以直接进行文件操作

由于redis容器里也没有相应的redis配置文件，因此先看下该redis镜像版本

root@2c7d734df8d4:/data# redis-server --version
Redis server v=4.0.9 sha=00000000:0 malloc=jemalloc-4.0.3 bits=64 build=22c6ee099916a58b

于是可以在主机上下载一下源码，找到对应版本的配置文件

lihui@docker:~$ wget http://download.redis.io/releases/redis-4.0.9.tar.gz

目录下有一个redis.conf配置文件，如果不放心，可以手动编译一下，如果缺少make，gcc，tcl可自行安装，build：

lihui@docker:~/redis-4.0.9$ make MALLOC=libc

对于redis-master，需要修改下面两个参数

# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize yes

和

# If a pid file is specified, Redis writes it where specified at startup
# and removes it at exit.
#
# When the server runs non daemonized, no pid file is created if none is
# specified in the configuration. When the server is daemonized, the pid file
# is used even if not specified, defaulting to "/var/run/redis.pid".
#
# Creating a pid file is best effort: if Redis is not able to create it
# nothing bad happens, the server will start and run normally.
pidfile /var/run/redis.pid

修改完之后，首先copy到挂载的volume目录

lihui@docker:~$ sudo cp redis.conf  /var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data/

在redis-master容器里，将该配置文件copy到执行目录下，启动redis-server即可

root@2c7d734df8d4:/data# cp redis.conf /usr/local/bin/
root@2c7d734df8d4:/data# cd /usr/local/bin/
root@2c7d734df8d4:/usr/local/bin# ./redis-server redis.conf
32:C 26 Apr 03:48:36.721 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
32:C 26 Apr 03:48:36.722 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=32, just started
32:C 26 Apr 03:48:36.723 # Configuration loaded
root@2c7d734df8d4:/usr/local/bin#
root@2c7d734df8d4:/usr/local/bin# ps aux | grep redis
root         33  0.0  0.3  41652  9188 ?        Ssl  03:48   0:00 ./redis-server 127.0.0.1:6379
root         38  0.0  0.0  11132  1000 pts/0    S+   03:48   0:00 grep redis

这样master就搞定了

redis-slave1容器配置

有的master操作，slave就简单了

先找到volume目

lihui@docker:~$ sudo docker inspect redis-slave1 | grep -i mounts -A 10
        "Mounts": [
            {
                "Type": "volume",
                "Name": "5b5cbb958fbaf62ae52158589f410e9c0d4f0b78d378138c12a8eba57da2e31e",
                "Source": "/var/lib/docker/volumes/5b5cbb958fbaf62ae52158589f410e9c0d4f0b78d378138c12a8eba57da2e31e/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }

修改redis.conf配置文件，注意除了master那两处修改之外，还要添加下面这行

################################# REPLICATION #################################

# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. A few things to understand ASAP about Redis replication.
#
# 1) Redis replication is asynchronous, but you can configure a master to
#    stop accepting writes if it appears to be not connected with at least
#    a given number of slaves.
# 2) Redis slaves are able to perform a partial resynchronization with the
#    master if the replication link is lost for a relatively small amount of
#    time. You may want to configure the replication backlog size (see the next
#    sections of this file) with a sensible value depending on your needs.
# 3) Replication is automatic and does not need user intervention. After a
#    network partition slaves automatically try to reconnect to masters
#    and resynchronize with them.
#
# slaveof 
slaveof master 6379

这个master是连接的name，而注释里是slaveof <masterip> <masterport>，这是因为slave启动的时候，指定了–link参数，会用连接名来代替具体的IP地址，这样在通信的时候，读取这里的master，也会替换成实际的IP地址；这里的端口号基本是默认

接着就同理

lihui@docker:~$ sudo cp redis.conf /var/lib/docker/volumes/5b5cbb958fbaf62ae52158589f410e9c0d4f0b78d378138c12a8eba57da2e31e/_data

redis-slave1容器启动redis

root@4b03b99660fb:/data# cp redis.conf /usr/local/bin/
root@4b03b99660fb:/data# cd /usr/local/bin/
root@4b03b99660fb:/usr/local/bin# ./redis-server redis.conf
9:C 26 Apr 04:00:23.713 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9:C 26 Apr 04:00:23.713 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=9, just started
9:C 26 Apr 04:00:23.713 # Configuration loaded
root@4b03b99660fb:/usr/local/bin# ps aux | grep redis
root         10  0.2  0.3  43748  9476 ?        Ssl  04:00   0:00 ./redis-server 127.0.0.1:6379
root         15  0.0  0.0  11008   504 pts/0    R+   04:00   0:00 grep redis

redis-slave2容器配置，和上面一样

lihui@docker:~$ sudo docker inspect redis-slave2 | grep -i mounts -A 10
        "Mounts": [
            {
                "Type": "volume",
                "Name": "67e50f0aac5d553eca847319a260318801c96bdb19f05f9ec067134683c71e84",
                "Source": "/var/lib/docker/volumes/67e50f0aac5d553eca847319a260318801c96bdb19f05f9ec067134683c71e84/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
lihui@docker:~$ sudo cp redis.conf /var/lib/docker/volumes/67e50f0aac5d553eca847319a260318801c96bdb19f05f9ec067134683c71e84/_data

启动redis

root@c16515555d20:/data# cp redis.conf /usr/local/bin/
root@c16515555d20:/data# cd /usr/local/bin/
root@c16515555d20:/usr/local/bin# ./redis-server redis.conf
11:C 26 Apr 04:03:31.150 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
11:C 26 Apr 04:03:31.150 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=11, just started
11:C 26 Apr 04:03:31.151 # Configuration loaded
root@c16515555d20:/usr/local/bin# ps aux | grep redis
root         12  0.0  0.3  43748  9472 ?        Ssl  04:03   0:00 ./redis-server 127.0.0.1:6379
root         17  0.0  0.0  11132   996 pts/0    S+   04:03   0:00 grep redis

redis测试

master里写入

root@2c7d734df8d4:/usr/local/bin# ./redis-cli
127.0.0.1:6379> set master lihui
OK
127.0.0.1:6379> get master
"lihui"

可是slave里没法同步到

root@4b03b99660fb:/usr/local/bin# ./redis-cli
127.0.0.1:6379> get master
(nil)
127.0.0.1:6379> get master
(nil)
127.0.0.1:6379> keys *
(empty list or set)

在slave容器里安装了telnet，发现到master是不通的；查看了下master里面redis-server进程，bind的IP地址是127.0.0.1，应该是这个问题

将redis.conf里配置的bind地址修改为0.0.0.0

# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 lookback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# bind 127.0.0.1
bind 0.0.0.0

重启启动后，看状态应该是正常了

root@2c7d734df8d4:/usr/local/bin# ps aux | grep redis
root        920  0.0  0.4  43700 11324 ?        Ssl  06:34   0:00 ./redis-server 0.0.0.0:6379
root        927  0.0  0.0  11132  1004 pts/0    S+   06:35   0:00 grep redis
root@2c7d734df8d4:/usr/local/bin#
root@2c7d734df8d4:/usr/local/bin#
root@2c7d734df8d4:/usr/local/bin# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.4,port=6379,state=online,offset=14,lag=1
slave1:ip=172.17.0.3,port=6379,state=online,offset=14,lag=1
master_replid:c5efd8a953283aaeef542149ebc48daefb7480dc
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:14
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:14
127.0.0.1:6379> get *
(nil)
127.0.0.1:6379> set master lihui
OK
127.0.0.1:6379> get master
"lihui"

从slave里获取信息无误

127.0.0.1:6379> get master
"lihui"
127.0.0.1:6379>

这里就将配置信息都修改下

最终Master数据库里的数据都可以同步到Slave数据库中，这样redis部分就搭建完了

APP容器Django配置

这个容器启动后，主要是访问数据库，提供WEB服务

首先Django容器里默认是已经安装了pip，通过pip安装python有关redis的支持包

root@e3f4df84474c:/# pip install redis

安装好了之后import试试是否成功，注意该镜像默认装的是python3

root@e3f4df84474c:/# python
Python 3.4.5 (default, Dec 14 2016, 18:54:20)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import redis
>>> print redis.__file__
  File "", line 1
    print redis.__file__
              ^
SyntaxError: Missing parentheses in call to 'print'
>>> print(redis.__file__)
/usr/local/lib/python3.4/site-packages/redis/__init__.py

这样可以通过python程序来调redis数据库了

下面就可以写WEB程序了；同样APP容器在启动的时候已经制定了挂载volume的目录，主机在~/Projects/Django/Appx，容器里在/usr/src/app

首先在容器该目录下创建APP

root@e3f4df84474c:/usr/src/app# mkdir dockerweb
root@e3f4df84474c:/usr/src/app# cd dockerweb/
root@e3f4df84474c:/usr/src/app/dockerweb# ls
root@e3f4df84474c:/usr/src/app/dockerweb# django-admin.py startproject redisweb
root@e3f4df84474c:/usr/src/app/dockerweb# ls
redisweb
root@e3f4df84474c:/usr/src/app/dockerweb# cd redisweb/
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# ls
manage.py  redisweb
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py startapp helloworld
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# ls
helloworld  manage.py  redisweb

主机进到该目录，helloworld应用

lihui@docker:~/Projects/Django/App1/dockerweb/redisweb/helloworld$ ls
__init__.py  admin.py  apps.py  migrations  models.py  tests.py  views.py

修改试图文件views.py

from django.shortcuts import render
from django.http import HttpResponse

# Create your views here.
import redis

def hello(request):
    str = redis.__file__
    str += "<br>"
    r = redis.Redis(host = "db", port = 6379, db = 0)
    info = r.info()
    str += ("Set Hi <br>")
    r.set("Hi", "HelloWorld-APP1")
    str += ("Get Hi: %s <br>" % r.get("Hi"))
    str += ("Redis Info: <br>")
    str += ("Key: Info Value")
    for key in info:
        str += ("%s: %s <br>" % (key, info[key]))
    return HttpResponse(str)

然后再redisweb/redisweb下，setting.py里安装选项添加helloworld

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'helloworld',
]

redisweb项目的URL模式文件urls.py，设置访问应用的URL模式，设置URL模式调用视图函数之间的映射表

from django.conf.urls import url
from django.contrib import admin
from helloworld.views import hello

urlpatterns = [
    url(r'^admin/', admin.site.urls),
    url(r'^helloworld$', hello),
]

修改完之后，进入到容器里，继续进行WEB项目的生成

root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py makemigrations
No changes detected
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying sessions.0001_initial... OK
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py syncdb
Unknown command: 'syncdb'
Type 'manage.py help' for usage.

这里报了一个错误，syncdb这个参数不对，经查询这个参数没了，搜了下Stack Overflow上说1.8版本以上的Django去除了，只需要前面两步即可

那么最后就启动该容器里的WEB服务，同时指定服务器的IP和端口；由于最终是通过HAProxy容器来接收外界网络IP地址的访问，实现负载均衡，这里APP1就用8001端口，地址0.0.0.0

root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8001
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 07:24:12
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8001/
Quit the server with CONTROL-C.

同理APP2也按上述修改，以端口8002启动

root@81382a640c1a:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8002
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 07:33:54
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8002/
Quit the server with CONTROL-C.

HAProxy容器配置

上面都完成之后，基本应用服务部署已经完毕了，HA只是一个负责负载均衡的代理容器作用，所有访问应用服务的都会经过HA来进行负载均衡

挂载volume隐射关系，主机在Projects/HAProxy，容器在/tmp目录

手动配置haproxy.cfg配置文件，注意listen端口是启动容器指定的端口号

lihui@docker:~/Projects/HAProxy$ sudo vim haproxy.cfg
lihui@docker:~/Projects/HAProxy$ cat haproxy.cfg
global
	log	127.0.0.1	local0
	maxconn	4096
	chroot	/usr/local/sbin
	daemon
	nbproc	4
	pidfile	/usr/local/sbin/haproxy.pid

defaults
	log	127.0.0.1	local3
	mode	http
	option	dontlognull
	option	redispatch
	retries	2
	maxconn	2000
	balance	roundrobin

	timeout connect 5000ms
	timeout client 50000ms
	timeout server 50000ms

listen redis_proxy 0.0.0.0:4444
	stats enable
	stats uri /haproxy-stats
		server APP1 APP1:8001 check inter 2000 rise 2 fall 5
		server APP2 APP2:8002 check inter 2000 rise 2 fall 5

将该配置文件copy到执行目录下，但是报了一个错误

root@f18e99c6ddef:/usr/local/sbin# cp /tmp/haproxy.cfg ./
root@f18e99c6ddef:/usr/local/sbin# haproxy -f haproxy.cfg
[ALERT] 115/074629 (13) : parsing [haproxy.cfg:22] : 'listen' cannot handle unexpected argument '0.0.0.0:4444'.
[ALERT] 115/074629 (13) : parsing [haproxy.cfg:22] : please use the 'bind' keyword for listening addresses.
[ALERT] 115/074629 (13) : Error(s) found in configuration file : haproxy.cfg
[ALERT] 115/074629 (13) : Fatal errors found in configuration.

看上去是listen那行不对，错误信息说要用bind，这个错误信息搜了半天，最终找到如下修改方法，启动就OK了

root@f18e99c6ddef:/usr/local/sbin# cp /tmp/haproxy.cfg ./
root@f18e99c6ddef:/usr/local/sbin# cat haproxy.cfg
global
	log	127.0.0.1	local0
	maxconn	4096
	chroot	/usr/local/sbin
	daemon
	nbproc	4
	pidfile	/usr/local/sbin/haproxy.pid

defaults
	log	127.0.0.1	local3
	mode	http
	option	dontlognull
	option	redispatch
	retries	2
	maxconn	2000
	balance	roundrobin

	timeout connect 5000ms
	timeout client 50000ms
	timeout server 50000ms

listen redis_proxy
	bind 0.0.0.0:4444
	stats enable
	stats uri /haproxy-stats
		server APP1 APP1:8001 check inter 2000 rise 2 fall 5
		server APP2 APP2:8002 check inter 2000 rise 2 fall 5
root@f18e99c6ddef:/usr/local/sbin#
root@f18e99c6ddef:/usr/local/sbin#
root@f18e99c6ddef:/usr/local/sbin# haproxy -f haproxy.cfg
[WARNING] 115/101457 (79) : Proxy 'redis_proxy': in multi-process mode, stats will be limited to process assigned to the current request.

到这里，基本所有的服务都配置结束了，下面就开始验证下APP正确性

玩耍WEB服务

由于HA所在的这个容器，启动的时候指定了参数-p 4444:4444，这样就映射了容器访问的端口到主机上，可以在主机上查看一下4444端口

lihui@docker:~/Projects/HAProxy$ sudo lsof -i:4444
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
docker-pr 8021 root    4u  IPv6 110960      0t0  TCP *:4444 (LISTEN)

如此一来，其它只要能连通本主机的，都可以通过本机IP地址和4444端口访问HA后面的WEB服务

先通过本机发送HTTP请求，看下返回情况，由于主机Linux是没有GUI的，因此就直接wget一下试试，HA所在容器的IP如果不想进去安装工具查询，可以主机上查看Linux进程，搜索端口号即可

lihui@docker:~/Projects/HAProxy$ ps aux | grep 4444
root       7982  0.0  0.1  52700  3820 pts/6    S+   Apr25   0:00 sudo docker run -it --name HAProxy --link APP1:APP1 --link APP2:APP2 -p 4444:4444 -v /home/lihui/Projects/HAProxy:/tmp haproxy /bin/bash
root       7983  0.0  1.0 313940 27040 pts/6    Sl+  Apr25   0:01 docker run -it --name HAProxy --link APP1:APP1 --link APP2:APP2 -p 4444:4444 -v /home/lihui/Projects/HAProxy:/tmp haproxy /bin/bash
root       8021  0.0  0.1 100816  3748 ?        Sl   Apr25   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4444 -container-ip 172.17.0.7 -container-port 4444
lihui     19475  0.0  0.0  14196   960 pts/3    S+   03:55   0:00 grep --color=auto 4444

如此可见HA的IP地址为172.17.0.7，wget一把，居然返回了400

lihui@docker:~/Projects/HAProxy$ wget http://172.17.0.7:4444/helloworld
--2018-04-26 03:57:16--  http://172.17.0.7:4444/helloworld
Connecting to 172.17.0.7:4444... connected.
HTTP request sent, awaiting response... 400 Bad Request
2018-04-26 03:57:16 ERROR 400: Bad Request.

看下两个APP的server端，啥错误原因，这里调度到APP2这里

root@81382a640c1a:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8002
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 10:22:26
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8002/
Quit the server with CONTROL-C.
Invalid HTTP_HOST header: '172.17.0.7:4444'. You may need to add '172.17.0.7' to ALLOWED_HOSTS.
[26/Apr/2018 10:57:16] "GET /helloworld HTTP/1.1" 400 58047

返回的信息是ALLOWED_HOSTS里没有添加白名单，那就照做一下，将两个APP容器里都添加以下HA的IP地址

文件为：dockerweb/redisweb/redisweb$ sudo vim settings.py，修改内容：

ALLOWED_HOSTS = ['172.17.0.7']

注意Web server要重新启动下

重启之后，再次在主机上wget一下，终于成功了

lihui@docker:~/Projects/HAProxy$ wget http://172.17.0.7:4444/helloworld
--2018-04-26 04:04:48--  http://172.17.0.7:4444/helloworld
Connecting to 172.17.0.7:4444... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
helloworld: Permission denied

Cannot write to 'helloworld' (Success).

这回Server端调度到APP1上了

root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8001
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 11:04:38
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8001/
Quit the server with CONTROL-C.
[26/Apr/2018 11:04:48] "GET /helloworld HTTP/1.1" 200 3375

由于我的Linux主机是MAC上的一个VMWare虚拟机，虽然没有GUI，但是我可以从MAC上来进行访问，由于MAC和VM之间是互通的，那么只需要将Linux主机的IP地址添加到APP后端Server的白名单，那么从MAC浏览器上访问Linux主机的IP地址，这样就会走到HA上，然后进行分发调度到最终其中一个APP Server上，返回结果

白名单加上Linux主机的IP地址

ALLOWED_HOSTS = ['172.17.0.7', '192.168.226.193']

最后，通过其它主机，这里是MAC，来进行浏览器访问VMware虚拟机的IP地址，转发端口4444，就自动映射到了HA容器服务，最终调度到其中一个WEB Server

NewImage

 

 

 

 

 

 

 

 

 

 

 

 

OVER

 

发表回复