Docker小试牛刀,集群部署简单WEB应用

要做的一件事,通过Docker部署一套WEB服务,用户进行访问HA节点进行负载均衡,进行分发调度到后端两个Server节点,一个Master数据库节点和两个Slave数据库节点,当然这里说的节点最终都部署为Docker容器,中间遇到了一些问题,记录一下

部署环境如下:

主机:VMWare Fusion虚拟机

操作系统:Ubuntu 16.04.4 LTS

内核版本:4.4.0-116-generic

Docker版本:Docker version 18.04.0-ce, build 3d479c0

大致结构图如下

NewImage

 

 

 

 

 

 

比较普通直接的流程,对外用户暴露的就是HA的IP地址和端口,这样用户通过访问HA,就会调度到后端WEB Server然后进行相应请求

先从Docker Hub里拉相关的镜像,具体包括

lihui@docker:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
haproxy             latest              2b6a4216580b        5 days ago          74.4MB
ubuntu              latest              c9d990395902        12 days ago         113MB
redis               latest              c5355f8853e4        4 weeks ago         107MB
django              latest              eb40dcf64078        16 months ago       436MB

容器启动顺序:redis-master => redis-slave => app => ha 

具体如下:

  1. 启动redis-master容器
  2. 两个redis-slave容器启动时连接到redis-master上
  3. 两个app容器启动时连接到redis-master上
  4. ha容器启动时连接到两个app上

启动容器

首先启动redis-master容器

lihui@docker:~$ sudo docker run -it --name redis-master redis /bin/bash
root@2c7d734df8d4:/data#

然后启动两个redis-slave容器

redis-slave1

lihui@docker:~$ sudo docker run -it --name redis-slave1 --link redis-master:master redis /bin/bash
root@4b03b99660fb:/data#

redis-slave2

lihui@docker:~$ sudo docker run -it --name redis-slave2 --link redis-master:master redis /bin/bash
root@c16515555d20:/data#

这里可以看到slave的启动和master稍有差别,多了一个–link选项,可以通过容器名来确定对应的容器

比如redis-slave1容器里,除了正常172.17.0.3和hostname的对应关系外,还记录了名称为master的连接信息,redis-master容器对应的IP地址为172.17.0.2

lihui@docker:~$ sudo docker run -it --name redis-slave1 --link redis-master:master redis /bin/bash
root@4b03b99660fb:/data# cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2	master 2c7d734df8d4 redis-master
172.17.0.3	4b03b99660fb

而redis-master容器里,却没有该类似信息

lihui@docker:~$ sudo docker run -it --name redis-master redis /bin/bash
root@2c7d734df8d4:/data# cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2	2c7d734df8d4

接着启动app容器,这里就是Django容器

APP1

lihui@docker:~$ sudo docker run -it --name APP1 --link redis-master:db -v ~/Projects/Django/App1:/usr/src/app django /bin/bash
root@e3f4df84474c:/#

APP2

lihui@docker:~$ sudo docker run -it --name APP2 --link redis-master:db -v ~/Projects/Django/App2:/usr/src/app django /bin/bash
root@81382a640c1a:/#

最后启动ha容器,端口挑一个没用到的

HAProxy

lihui@docker:~$ sudo docker run -it --name HAProxy --link APP1:APP1 --link APP2:APP2 -p 4444:4444 -v ~/Projects/HAProxy:/tmp haproxy /bin/bash
root@f18e99c6ddef:/#

这样,相关的容器都已经启动完毕了,查看一下

lihui@docker:~$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                    NAMES
f18e99c6ddef        haproxy             "/docker-entrypoint.…"   About a minute ago   Up About a minute   0.0.0.0:4444->4444/tcp   HAProxy
81382a640c1a        django              "/bin/bash"              7 minutes ago        Up 7 minutes                                 APP2
e3f4df84474c        django              "/bin/bash"              9 minutes ago        Up 9 minutes                                 APP1
c16515555d20        redis               "docker-entrypoint.s…"   20 minutes ago       Up 20 minutes       6379/tcp                 redis-slave2
4b03b99660fb        redis               "docker-entrypoint.s…"   21 minutes ago       Up 21 minutes       6379/tcp                 redis-slave1
2c7d734df8d4        redis               "docker-entrypoint.s…"   23 minutes ago       Up 23 minutes       6379/tcp                 redis-master

查看veth pair的个数,也是6个,和容器数一致

lihui@docker:~$ ip l | grep veth | wc -l
6

更新配置

当前只是启动了容器,而如果要正确地启动每个服务,需要根据自己的配置进行更新,才能完成通信以及相应的功能

由于容器本身就是轻量级,不可能像一个KVM一个Linux ISO那样,携带一些常用工具,正常连文本编辑器也没有,当然如果你想联网安装对应的软件包或者上传安装包也是可以的,但是其实有种更简单的方法,通过volume来实现创建文件;在容器启动时,指定-v参数挂载volume,,这样主机和容器之间就可以共享数据,就可以直接主机上进行操作;上面启动容器的时候可以看到redis并没有指定volume,原因是redis镜像里已经集成了volume挂载命令,因此在启动后就已经挂载好了,直接用即可

redis-master容器配置

先在host节点上查看一下挂载的volume在哪里

lihui@docker:~$ sudo docker inspect redis-master | grep -i mounts -A 10
        "Mounts": [
            {
                "Type": "volume",
                "Name": "23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0",
                "Source": "/var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }

可见主机目录为/var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data,对应着容器的目录为/data

可以先做一下简单的测试,主机该目录写一个文件

lihui@docker:~$ sudo touch /var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data/hi
lihui@docker:~$ sudo ls -l /var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data/
total 0
-rw-r--r-- 1 root root 0 Apr 25 20:13 hi

查看容器里面

root@2c7d734df8d4:/data# ls -l
total 0
-rw-r--r-- 1 root root 0 Apr 26 03:13 hi

这里时区没对上,先不用管,可见可以直接进行文件操作

由于redis容器里也没有相应的redis配置文件,因此先看下该redis镜像版本

root@2c7d734df8d4:/data# redis-server --version
Redis server v=4.0.9 sha=00000000:0 malloc=jemalloc-4.0.3 bits=64 build=22c6ee099916a58b

于是可以在主机上下载一下源码,找到对应版本的配置文件

lihui@docker:~$ wget http://download.redis.io/releases/redis-4.0.9.tar.gz

目录下有一个redis.conf配置文件,如果不放心,可以手动编译一下,如果缺少make,gcc,tcl可自行安装,build:

lihui@docker:~/redis-4.0.9$ make MALLOC=libc

对于redis-master,需要修改下面两个参数

# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize yes

# If a pid file is specified, Redis writes it where specified at startup
# and removes it at exit.
#
# When the server runs non daemonized, no pid file is created if none is
# specified in the configuration. When the server is daemonized, the pid file
# is used even if not specified, defaulting to "/var/run/redis.pid".
#
# Creating a pid file is best effort: if Redis is not able to create it
# nothing bad happens, the server will start and run normally.
pidfile /var/run/redis.pid

修改完之后,首先copy到挂载的volume目录

lihui@docker:~$ sudo cp redis.conf  /var/lib/docker/volumes/23428bc7b45a780823179e1ef57374e5d6f04199994c1e936cba2e08788ddaa0/_data/

在redis-master容器里,将该配置文件copy到执行目录下,启动redis-server即可

root@2c7d734df8d4:/data# cp redis.conf /usr/local/bin/
root@2c7d734df8d4:/data# cd /usr/local/bin/
root@2c7d734df8d4:/usr/local/bin# ./redis-server redis.conf
32:C 26 Apr 03:48:36.721 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
32:C 26 Apr 03:48:36.722 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=32, just started
32:C 26 Apr 03:48:36.723 # Configuration loaded
root@2c7d734df8d4:/usr/local/bin#
root@2c7d734df8d4:/usr/local/bin# ps aux | grep redis
root         33  0.0  0.3  41652  9188 ?        Ssl  03:48   0:00 ./redis-server 127.0.0.1:6379
root         38  0.0  0.0  11132  1000 pts/0    S+   03:48   0:00 grep redis

这样master就搞定了

redis-slave1容器配置

有的master操作,slave就简单了

先找到volume目

lihui@docker:~$ sudo docker inspect redis-slave1 | grep -i mounts -A 10
        "Mounts": [
            {
                "Type": "volume",
                "Name": "5b5cbb958fbaf62ae52158589f410e9c0d4f0b78d378138c12a8eba57da2e31e",
                "Source": "/var/lib/docker/volumes/5b5cbb958fbaf62ae52158589f410e9c0d4f0b78d378138c12a8eba57da2e31e/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }

修改redis.conf配置文件,注意除了master那两处修改之外,还要添加下面这行

################################# REPLICATION #################################

# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. A few things to understand ASAP about Redis replication.
#
# 1) Redis replication is asynchronous, but you can configure a master to
#    stop accepting writes if it appears to be not connected with at least
#    a given number of slaves.
# 2) Redis slaves are able to perform a partial resynchronization with the
#    master if the replication link is lost for a relatively small amount of
#    time. You may want to configure the replication backlog size (see the next
#    sections of this file) with a sensible value depending on your needs.
# 3) Replication is automatic and does not need user intervention. After a
#    network partition slaves automatically try to reconnect to masters
#    and resynchronize with them.
#
# slaveof 
slaveof master 6379

这个master是连接的name,而注释里是slaveof <masterip> <masterport>,这是因为slave启动的时候,指定了–link参数,会用连接名来代替具体的IP地址,这样在通信的时候,读取这里的master,也会替换成实际的IP地址;这里的端口号基本是默认

接着就同理

lihui@docker:~$ sudo cp redis.conf /var/lib/docker/volumes/5b5cbb958fbaf62ae52158589f410e9c0d4f0b78d378138c12a8eba57da2e31e/_data

redis-slave1容器启动redis

root@4b03b99660fb:/data# cp redis.conf /usr/local/bin/
root@4b03b99660fb:/data# cd /usr/local/bin/
root@4b03b99660fb:/usr/local/bin# ./redis-server redis.conf
9:C 26 Apr 04:00:23.713 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9:C 26 Apr 04:00:23.713 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=9, just started
9:C 26 Apr 04:00:23.713 # Configuration loaded
root@4b03b99660fb:/usr/local/bin# ps aux | grep redis
root         10  0.2  0.3  43748  9476 ?        Ssl  04:00   0:00 ./redis-server 127.0.0.1:6379
root         15  0.0  0.0  11008   504 pts/0    R+   04:00   0:00 grep redis

redis-slave2容器配置,和上面一样

lihui@docker:~$ sudo docker inspect redis-slave2 | grep -i mounts -A 10
        "Mounts": [
            {
                "Type": "volume",
                "Name": "67e50f0aac5d553eca847319a260318801c96bdb19f05f9ec067134683c71e84",
                "Source": "/var/lib/docker/volumes/67e50f0aac5d553eca847319a260318801c96bdb19f05f9ec067134683c71e84/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
lihui@docker:~$ sudo cp redis.conf /var/lib/docker/volumes/67e50f0aac5d553eca847319a260318801c96bdb19f05f9ec067134683c71e84/_data

启动redis

root@c16515555d20:/data# cp redis.conf /usr/local/bin/
root@c16515555d20:/data# cd /usr/local/bin/
root@c16515555d20:/usr/local/bin# ./redis-server redis.conf
11:C 26 Apr 04:03:31.150 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
11:C 26 Apr 04:03:31.150 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=11, just started
11:C 26 Apr 04:03:31.151 # Configuration loaded
root@c16515555d20:/usr/local/bin# ps aux | grep redis
root         12  0.0  0.3  43748  9472 ?        Ssl  04:03   0:00 ./redis-server 127.0.0.1:6379
root         17  0.0  0.0  11132   996 pts/0    S+   04:03   0:00 grep redis

redis测试

master里写入

root@2c7d734df8d4:/usr/local/bin# ./redis-cli
127.0.0.1:6379> set master lihui
OK
127.0.0.1:6379> get master
"lihui"

可是slave里没法同步到

root@4b03b99660fb:/usr/local/bin# ./redis-cli
127.0.0.1:6379> get master
(nil)
127.0.0.1:6379> get master
(nil)
127.0.0.1:6379> keys *
(empty list or set)

在slave容器里安装了telnet,发现到master是不通的;查看了下master里面redis-server进程,bind的IP地址是127.0.0.1,应该是这个问题

将redis.conf里配置的bind地址修改为0.0.0.0

# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 lookback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# bind 127.0.0.1
bind 0.0.0.0

重启启动后,看状态应该是正常了

root@2c7d734df8d4:/usr/local/bin# ps aux | grep redis
root        920  0.0  0.4  43700 11324 ?        Ssl  06:34   0:00 ./redis-server 0.0.0.0:6379
root        927  0.0  0.0  11132  1004 pts/0    S+   06:35   0:00 grep redis
root@2c7d734df8d4:/usr/local/bin#
root@2c7d734df8d4:/usr/local/bin#
root@2c7d734df8d4:/usr/local/bin# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.4,port=6379,state=online,offset=14,lag=1
slave1:ip=172.17.0.3,port=6379,state=online,offset=14,lag=1
master_replid:c5efd8a953283aaeef542149ebc48daefb7480dc
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:14
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:14
127.0.0.1:6379> get *
(nil)
127.0.0.1:6379> set master lihui
OK
127.0.0.1:6379> get master
"lihui"

从slave里获取信息无误

127.0.0.1:6379> get master
"lihui"
127.0.0.1:6379>

这里就将配置信息都修改下

最终Master数据库里的数据都可以同步到Slave数据库中,这样redis部分就搭建完了

APP容器Django配置

这个容器启动后,主要是访问数据库,提供WEB服务

首先Django容器里默认是已经安装了pip,通过pip安装python有关redis的支持包

root@e3f4df84474c:/# pip install redis

安装好了之后import试试是否成功,注意该镜像默认装的是python3

root@e3f4df84474c:/# python
Python 3.4.5 (default, Dec 14 2016, 18:54:20)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import redis
>>> print redis.__file__
  File "", line 1
    print redis.__file__
              ^
SyntaxError: Missing parentheses in call to 'print'
>>> print(redis.__file__)
/usr/local/lib/python3.4/site-packages/redis/__init__.py

这样可以通过python程序来调redis数据库了

下面就可以写WEB程序了;同样APP容器在启动的时候已经制定了挂载volume的目录,主机在~/Projects/Django/Appx,容器里在/usr/src/app

首先在容器该目录下创建APP

root@e3f4df84474c:/usr/src/app# mkdir dockerweb
root@e3f4df84474c:/usr/src/app# cd dockerweb/
root@e3f4df84474c:/usr/src/app/dockerweb# ls
root@e3f4df84474c:/usr/src/app/dockerweb# django-admin.py startproject redisweb
root@e3f4df84474c:/usr/src/app/dockerweb# ls
redisweb
root@e3f4df84474c:/usr/src/app/dockerweb# cd redisweb/
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# ls
manage.py  redisweb
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py startapp helloworld
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# ls
helloworld  manage.py  redisweb

主机进到该目录,helloworld应用

lihui@docker:~/Projects/Django/App1/dockerweb/redisweb/helloworld$ ls
__init__.py  admin.py  apps.py  migrations  models.py  tests.py  views.py

修改试图文件views.py

from django.shortcuts import render
from django.http import HttpResponse

# Create your views here.
import redis

def hello(request):
str = redis.__file__
str += "<br>"
r = redis.Redis(host = "db", port = 6379, db = 0)
info = r.info()
str += ("Set Hi <br>")
r.set("Hi", "HelloWorld-APP1")
str += ("Get Hi: %s <br>" % r.get("Hi"))
str += ("Redis Info: <br>")
str += ("Key: Info Value")
for key in info:
str += ("%s: %s <br>" % (key, info[key]))
return HttpResponse(str)

然后再redisweb/redisweb下,setting.py里安装选项添加helloworld

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'helloworld',
]

redisweb项目的URL模式文件urls.py,设置访问应用的URL模式,设置URL模式调用视图函数之间的映射表

from django.conf.urls import url
from django.contrib import admin
from helloworld.views import hello

urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^helloworld$', hello),
]

修改完之后,进入到容器里,继续进行WEB项目的生成

root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py makemigrations
No changes detected
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying sessions.0001_initial... OK
root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py syncdb
Unknown command: 'syncdb'
Type 'manage.py help' for usage.

这里报了一个错误,syncdb这个参数不对,经查询这个参数没了,搜了下Stack Overflow上说1.8版本以上的Django去除了,只需要前面两步即可

那么最后就启动该容器里的WEB服务,同时指定服务器的IP和端口;由于最终是通过HAProxy容器来接收外界网络IP地址的访问,实现负载均衡,这里APP1就用8001端口,地址0.0.0.0

root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8001
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 07:24:12
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8001/
Quit the server with CONTROL-C.

同理APP2也按上述修改,以端口8002启动

root@81382a640c1a:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8002
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 07:33:54
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8002/
Quit the server with CONTROL-C.

HAProxy容器配置

上面都完成之后,基本应用服务部署已经完毕了,HA只是一个负责负载均衡的代理容器作用,所有访问应用服务的都会经过HA来进行负载均衡

挂载volume隐射关系,主机在Projects/HAProxy,容器在/tmp目录

手动配置haproxy.cfg配置文件,注意listen端口是启动容器指定的端口号

lihui@docker:~/Projects/HAProxy$ sudo vim haproxy.cfg
lihui@docker:~/Projects/HAProxy$ cat haproxy.cfg
global
	log	127.0.0.1	local0
	maxconn	4096
	chroot	/usr/local/sbin
	daemon
	nbproc	4
	pidfile	/usr/local/sbin/haproxy.pid

defaults
	log	127.0.0.1	local3
	mode	http
	option	dontlognull
	option	redispatch
	retries	2
	maxconn	2000
	balance	roundrobin

	timeout connect 5000ms
	timeout client 50000ms
	timeout server 50000ms

listen redis_proxy 0.0.0.0:4444
	stats enable
	stats uri /haproxy-stats
		server APP1 APP1:8001 check inter 2000 rise 2 fall 5
		server APP2 APP2:8002 check inter 2000 rise 2 fall 5

将该配置文件copy到执行目录下,但是报了一个错误

root@f18e99c6ddef:/usr/local/sbin# cp /tmp/haproxy.cfg ./
root@f18e99c6ddef:/usr/local/sbin# haproxy -f haproxy.cfg
[ALERT] 115/074629 (13) : parsing [haproxy.cfg:22] : 'listen' cannot handle unexpected argument '0.0.0.0:4444'.
[ALERT] 115/074629 (13) : parsing [haproxy.cfg:22] : please use the 'bind' keyword for listening addresses.
[ALERT] 115/074629 (13) : Error(s) found in configuration file : haproxy.cfg
[ALERT] 115/074629 (13) : Fatal errors found in configuration.

看上去是listen那行不对,错误信息说要用bind,这个错误信息搜了半天,最终找到如下修改方法,启动就OK了

root@f18e99c6ddef:/usr/local/sbin# cp /tmp/haproxy.cfg ./
root@f18e99c6ddef:/usr/local/sbin# cat haproxy.cfg
global
	log	127.0.0.1	local0
	maxconn	4096
	chroot	/usr/local/sbin
	daemon
	nbproc	4
	pidfile	/usr/local/sbin/haproxy.pid

defaults
	log	127.0.0.1	local3
	mode	http
	option	dontlognull
	option	redispatch
	retries	2
	maxconn	2000
	balance	roundrobin

	timeout connect 5000ms
	timeout client 50000ms
	timeout server 50000ms

listen redis_proxy
	bind 0.0.0.0:4444
	stats enable
	stats uri /haproxy-stats
		server APP1 APP1:8001 check inter 2000 rise 2 fall 5
		server APP2 APP2:8002 check inter 2000 rise 2 fall 5
root@f18e99c6ddef:/usr/local/sbin#
root@f18e99c6ddef:/usr/local/sbin#
root@f18e99c6ddef:/usr/local/sbin# haproxy -f haproxy.cfg
[WARNING] 115/101457 (79) : Proxy 'redis_proxy': in multi-process mode, stats will be limited to process assigned to the current request.

到这里,基本所有的服务都配置结束了,下面就开始验证下APP正确性

玩耍WEB服务

由于HA所在的这个容器,启动的时候指定了参数-p 4444:4444,这样就映射了容器访问的端口到主机上,可以在主机上查看一下4444端口

lihui@docker:~/Projects/HAProxy$ sudo lsof -i:4444
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
docker-pr 8021 root    4u  IPv6 110960      0t0  TCP *:4444 (LISTEN)

如此一来,其它只要能连通本主机的,都可以通过本机IP地址和4444端口访问HA后面的WEB服务

先通过本机发送HTTP请求,看下返回情况,由于主机Linux是没有GUI的,因此就直接wget一下试试,HA所在容器的IP如果不想进去安装工具查询,可以主机上查看Linux进程,搜索端口号即可

lihui@docker:~/Projects/HAProxy$ ps aux | grep 4444
root       7982  0.0  0.1  52700  3820 pts/6    S+   Apr25   0:00 sudo docker run -it --name HAProxy --link APP1:APP1 --link APP2:APP2 -p 4444:4444 -v /home/lihui/Projects/HAProxy:/tmp haproxy /bin/bash
root       7983  0.0  1.0 313940 27040 pts/6    Sl+  Apr25   0:01 docker run -it --name HAProxy --link APP1:APP1 --link APP2:APP2 -p 4444:4444 -v /home/lihui/Projects/HAProxy:/tmp haproxy /bin/bash
root       8021  0.0  0.1 100816  3748 ?        Sl   Apr25   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4444 -container-ip 172.17.0.7 -container-port 4444
lihui     19475  0.0  0.0  14196   960 pts/3    S+   03:55   0:00 grep --color=auto 4444

如此可见HA的IP地址为172.17.0.7,wget一把,居然返回了400

lihui@docker:~/Projects/HAProxy$ wget http://172.17.0.7:4444/helloworld
--2018-04-26 03:57:16--  http://172.17.0.7:4444/helloworld
Connecting to 172.17.0.7:4444... connected.
HTTP request sent, awaiting response... 400 Bad Request
2018-04-26 03:57:16 ERROR 400: Bad Request.

看下两个APP的server端,啥错误原因,这里调度到APP2这里

root@81382a640c1a:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8002
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 10:22:26
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8002/
Quit the server with CONTROL-C.
Invalid HTTP_HOST header: '172.17.0.7:4444'. You may need to add '172.17.0.7' to ALLOWED_HOSTS.
[26/Apr/2018 10:57:16] "GET /helloworld HTTP/1.1" 400 58047

返回的信息是ALLOWED_HOSTS里没有添加白名单,那就照做一下,将两个APP容器里都添加以下HA的IP地址

文件为:dockerweb/redisweb/redisweb$ sudo vim settings.py,修改内容:

ALLOWED_HOSTS = ['172.17.0.7']

注意Web server要重新启动下

重启之后,再次在主机上wget一下,终于成功了

lihui@docker:~/Projects/HAProxy$ wget http://172.17.0.7:4444/helloworld
--2018-04-26 04:04:48--  http://172.17.0.7:4444/helloworld
Connecting to 172.17.0.7:4444... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
helloworld: Permission denied

Cannot write to 'helloworld' (Success).

这回Server端调度到APP1上了

root@e3f4df84474c:/usr/src/app/dockerweb/redisweb# python manage.py runserver 0.0.0.0:8001
Performing system checks...

System check identified no issues (0 silenced).
April 26, 2018 - 11:04:38
Django version 1.10.4, using settings 'redisweb.settings'
Starting development server at http://0.0.0.0:8001/
Quit the server with CONTROL-C.
[26/Apr/2018 11:04:48] "GET /helloworld HTTP/1.1" 200 3375

由于我的Linux主机是MAC上的一个VMWare虚拟机,虽然没有GUI,但是我可以从MAC上来进行访问,由于MAC和VM之间是互通的,那么只需要将Linux主机的IP地址添加到APP后端Server的白名单,那么从MAC浏览器上访问Linux主机的IP地址,这样就会走到HA上,然后进行分发调度到最终其中一个APP Server上,返回结果

白名单加上Linux主机的IP地址

ALLOWED_HOSTS = ['172.17.0.7', '192.168.226.193']

最后,通过其它主机,这里是MAC,来进行浏览器访问VMware虚拟机的IP地址,转发端口4444,就自动映射到了HA容器服务,最终调度到其中一个WEB Server

NewImage

 

 

 

 

 

 

 

 

 

 

 

 

OVER

 

发表回复