|
前文我們聊了下haproxy的訪問控制ACL的配置，回顧請參考https://www.cnblogs.com/qiuhom-1874/p/12817773.html；今天我們來聊一聊haproxy的https的配置、基于tcp四層負載均衡以及訪問控制的配置說明； 首先我們來看一下haproxy的https的配置；https是什么我這里就不過多闡述了，有關(guān)證書的申請相關(guān)說明請參考https://www.cnblogs.com/qiuhom-1874/p/12237944.html；在haproxy的配置文件中，我們要明確的聲明監(jiān)聽某個端口，該端口需要ssl協(xié)議來訪問，類似nginx里的listen 443 ssl配置；除此之外我們還需要用crt來指定證書；不同于nginx里的配置是haproxy的證書內(nèi)容包含私鑰信息；所以在我們申請好證書后，還需要把證書文件內(nèi)容同私鑰文件做合并； 示例：配置haproxy監(jiān)聽443端口，并支持https
提示：紅框中的內(nèi)容表示監(jiān)聽443端口，并明確指定使用ssl協(xié)議訪問；證書文件是/etc/haproxy/ssl/haproxy.pem 創(chuàng)建ssl目錄和證書文件 [root@docker_node1 ~]# mkdir -p /etc/haproxy/ssl [root@docker_node1 ~]# cd /etc/haproxy/ssl [root@docker_node1 ssl]# ls [root@docker_node1 ssl]# (umask 066;openssl genrsa -out haproxy.key 2048) Generating RSA private key, 2048 bit long modulus .......+++ ...+++ e is 65537 (0x10001) [root@docker_node1 ssl]# ls haproxy.key [root@docker_node1 ssl]# openssl req -new -x509 -key haproxy.key -out haproxy.crt -subj "/CN=www.test.com" [root@docker_node1 ssl]# ls haproxy.crt haproxy.key [root@docker_node1 ssl]# cat haproxy.crt haproxy.key > haproxy.pem [root@docker_node1 ssl]# ls haproxy.crt haproxy.key haproxy.pem [root@docker_node1 ssl]# openssl x509 -in haproxy.pem -noout -text Certificate: Data: Version: 3 (0x2) Serial Number: e6:c5:30:f9:10:e5:da:cf Signature Algorithm: sha256WithRSAEncryption Issuer: CN=www.test.com Validity Not Before: May 2 14:41:50 2020 GMT Not After : Jun 1 14:41:50 2020 GMT Subject: CN=www.test.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:e6:bd:db:2d:37:1e:b3:26:21:45:37:c3:bc:c2: f8:f0:35:31:7c:af:f4:22:be:bc:f2:01:a1:ce:31: 32:91:37:06:14:a8:eb:ec:99:30:ae:8f:66:6b:51: 03:94:bb:d1:ae:7d:15:23:ea:9a:83:74:6d:d0:be: 30:5d:bf:4c:dd:79:c6:8d:51:01:ea:d7:a8:e5:93: f9:11:d9:75:cc:2c:65:d6:31:db:15:20:7f:5e:9f: 56:15:3e:17:b9:82:a8:25:ba:40:17:1c:ef:f2:fc: 11:cb:72:ce:07:5b:57:5d:c3:f1:f6:42:1f:02:63: b0:33:de:87:a3:43:c7:a0:1b:03:c4:ab:09:f4:67: ce:ec:ef:1f:88:7a:c2:aa:ca:41:ed:1d:78:60:b1: 41:6d:b9:46:67:1e:c9:ce:43:d8:d6:36:be:ba:ed: d9:2b:01:9a:0c:8e:64:59:07:3a:c2:2c:55:2b:9b: 06:e5:5b:c7:ba:e2:9f:ea:be:71:9f:76:da:79:c6: f0:b8:94:70:c0:4f:7b:57:53:5c:30:71:c5:82:d0: 7a:a8:d0:71:32:2c:f0:38:61:69:9b:9e:ac:da:45: 74:63:25:76:78:91:c3:be:6c:7c:72:1c:54:4a:ac: 10:24:45:9c:89:6a:1f:5b:00:22:3a:b6:fe:3f:b8: 73:7b Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: B2:EC:53:AE:68:E0:65:1D:E6:DB:C8:16:E3:BA:D3:70:BC:E7:79:81 X509v3 Authority Key Identifier: keyid:B2:EC:53:AE:68:E0:65:1D:E6:DB:C8:16:E3:BA:D3:70:BC:E7:79:81 X509v3 Basic Constraints: CA:TRUE Signature Algorithm: sha256WithRSAEncryption 91:a2:36:9b:b8:d3:fd:89:ea:6c:b4:b9:3c:b6:a1:f3:86:87: 71:13:8b:10:a0:73:e9:96:b7:1b:dd:7e:91:64:41:a4:c3:80: 7c:b6:f2:ce:a7:77:28:74:51:da:a6:52:98:a7:88:48:41:2f: ad:3d:cb:36:bd:94:f9:27:72:f1:e8:ae:f8:bd:38:2e:d6:ea: 0b:26:2d:8b:49:3e:bc:4e:58:9e:bf:79:99:5b:88:23:da:98: e4:45:79:9c:d2:c9:05:05:9f:23:8a:0f:38:db:9a:c4:5c:e9: a0:f5:e5:aa:02:bc:df:a7:5d:11:cd:35:08:cc:41:08:40:b6: 0b:e4:ad:79:df:9a:1d:7b:70:8b:65:e5:dc:85:34:55:5c:f2: 94:dc:07:91:43:d0:bb:b4:fb:31:b9:74:19:7a:69:43:11:70: 41:41:86:93:ad:83:42:62:e2:67:69:38:cd:18:c7:e9:f7:f4: be:78:22:ea:ee:20:db:27:1f:06:87:4c:51:67:19:0a:64:97: 3a:e6:2c:32:bd:84:91:88:96:d2:01:e5:c3:62:59:11:c8:20: 7d:a1:c8:5a:3d:8a:fc:f5:14:fe:41:15:97:ee:47:ec:e5:19: 49:0b:c1:8a:c9:3f:10:4d:66:bf:d5:01:21:2d:fd:8b:a7:95: 17:08:7e:46 [root@docker_node1 ssl]# 提示：我這里就直接用-subj來指定證書/CN來生成自簽名證書；haproxy的證書要求是把證書信息和私鑰信息放在一起，所以我們需要把證書文件信息和私鑰新信息通過重定向的方式合并在一起； 測試：重啟haproxy,用瀏覽器訪問
提示：可以看到我們用https訪問是可以正常的訪問，只是證書瀏覽器不信任，所以會提示我們不安全； 把80端口的請求重向定443
提示：紅框中的配置表示把http請求重定向為https，這個類似nginx里的重寫url； 測試：用瀏覽器訪問http://192.168.0.22看看是否會把http重寫為https?
提示：可以看到我們訪問用http去訪問，在響應(yīng)報文中會用location告訴瀏覽器去訪問https://192.168.0.22；這樣一來就實現(xiàn)了全站https； 向后端傳遞用戶請求的協(xié)議和端口（frontend或backend）
提示：紅框中的內(nèi)容就表示向后端請求報文中設(shè)置X-Forwarded-port首部的值為dst_port變量的值；添加X-Forwared-Proto首部，其值為https，在請求協(xié)議是https的請求時；通常這兩個設(shè)置是后端應(yīng)用server要求使用https訪問時需要把前端https請求通過X-Forwared-Proto傳遞給后端； 測試：在后端server上配置日志格式，分別將這兩個首部記錄到日志，然后通過訪問，查看日志中記錄這兩個首部的值是否是我們訪問的443端口和https協(xié)議？
提示：以上定義httpd的日志格式，分別記錄{X-Forwarded-Port和X-Forwared-Proto首部的值 用瀏覽器訪問測試，看看日志中記錄的信息
提示：可以看到日志中把對應(yīng)訪問端口和協(xié)議都傳給了后端server； haproxy基于4層代理做tcp負載均衡 示例：基于tcp對mysql做負載均衡
提示：以上配置表示用tcp協(xié)議來代理后端server； 后端server環(huán)境準(zhǔn)備 [root@docker_node1 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE wordpress latest c3fa1c8546fb 44 hours ago 540MB mysql 5.7 f965319e89de 4 days ago 448MB httpd 2.4.37-alpine dfd436f9a5d8 16 months ago 91.8MB [root@docker_node1 ~]# docker run --name db1 -d --net bridge -e MYSQL_ROOT_PASSWORD=admin mysql:5.7 8cade03e4f28f40e7d7e970355735c8b881892754b18a637fef0d956f3f88877 [root@docker_node1 ~]# docker run --name db2 -d --net bridge -e MYSQL_ROOT_PASSWORD=admin mysql:5.7 1bff841c226abe88441103764655dc86257d5b99079eaa113384fd04b62cf0f8 [root@docker_node1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1bff841c226a mysql:5.7 "docker-entrypoint.s…" 7 seconds ago Up 6 seconds 3306/tcp, 33060/tcp db2 8cade03e4f28 mysql:5.7 "docker-entrypoint.s…" 12 seconds ago Up 12 seconds 3306/tcp, 33060/tcp db1 3572c621f827 wordpress "docker-entrypoint.s…" 9 hours ago Up 9 hours 80/tcp wordpress f07288d607e5 httpd:2.4.37-alpine "httpd-foreground" 7 days ago Up 13 hours 80/tcp web3 971595b7f409 httpd:2.4.37-alpine "httpd-foreground" 7 days ago Up 13 hours 80/tcp web2 5c74f3be1868 httpd:2.4.37-alpine "httpd-foreground" 7 days ago Up 13 hours 80/tcp web1 [root@docker_node1 ~]# docker exec -it db1 /bin/sh # mysql -padmin mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.30 MySQL Community Server (GPL) Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant all on *.* to "myuser"@'172.17.0.%' identified by 'admin'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> \q Bye # exit [root@docker_node1 ~]# docker exec -it db2 /bin/sh # mysql -padmin mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.30 MySQL Community Server (GPL) Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant all on *.* to "myuser"@'172.17.0.%' identified by 'admin'; Query OK, 0 rows affected, 1 warning (0.01 sec) mysql> \q Bye # exit [root@docker_node1 ~]# 提示：啟動mysql容器時，我們需要對root設(shè)置密碼；新建用戶時，我們指定的主機IP地址應(yīng)該是docker0橋的地址，雖然是基于4層tcp代理，但它還是要更改源ip； 測試：重啟haproxy，用mysql客戶端訪問haproxy所監(jiān)聽的3306端口，看看是否能夠正常的連接到后端mysql容器里？
提示：可以看到用別的主機上的mysql客戶端是可以正常訪問haproxy監(jiān)聽的3306端口； haproxy基于4層tcp做訪問控制
提示：tcp-request connection 表示根據(jù)第4層條件對傳入連接執(zhí)行操作；以上配置表示拒絕源地址為192.168.0.21的連接 測試：重啟haproxy，用192.168.0.21上的mysql客戶端連接192.168.0.22：3306看看是否能夠正常連接？
提示：可以看到在192.168.0.21上用mysql工具就不能夠連接192.168.0.22上的mysql了； 示例：僅開放源地址為192.168.0.21的主機訪問，拒絕其他主機的訪問
提示：tcp-request connection 允許訪問用accept來標(biāo)識，拒絕用reject；如果是聯(lián)合使用后面范圍大的可以不用寫條件，表示除上面匹配到的ACL條目以外的所有條目；同http-request 類似，不同的是http-request允許是用allow標(biāo)識，拒絕用deny標(biāo)識； 測試：分別用源地址為192.168.0.21的主機上的mysql客戶端去連接192.168.0.22上的mysql和源地址非192.168.0.21上的mysql客戶端連接192.168.0.22上的mysql看看是否都可以連接？
提示：可以看到在源地址非192.168.0.21上使用mysql客戶端是不能夠連接到后端mysql容器的；這是因為它沒有被我們定義ACL匹配，所以就會被tcp-request connect reject匹配；從而拒絕連接； 最后一個話題，在前邊的博客中我們聊的了haproxy的狀態(tài)頁的配置；其中我們演示的是通過頁面展示給我們，通過鼠標(biāo)點點點去管理后端server；這樣一來在對于我們想要監(jiān)控haproxy本身以及后端server就是一個難題；聰明的你一定注意到haproxy的全局配置段中有一個stat socket /var/lib/haproxy/stats這樣的配置；我們接下來說說這個配置是干嘛用的；
提示：以上配置是表示將監(jiān)控頁面信息綁定到一個socket文件上；我們可以通過對該socket文件發(fā)送特定指令，實現(xiàn)操作haproxy的目的 示例：通過socat向/var/lib/haproxy/stats傳遞help信息，讓其打印幫助頁面 [root@docker_node1 ~]# echo "help"|socat stdio /var/lib/haproxy/stats Unknown command. Please enter one of the following commands only : clear counters : clear max statistics counters (add 'all' for all counters) clear table : remove an entry from a table help : this message prompt : toggle interactive mode with prompt quit : disconnect show info : report information about the running process show pools : report information about the memory pools usage show stat : report counters for each proxy and server show errors : report last request and response errors for each proxy show sess [id] : report the list of current sessions or dump this session show table [id]: report table usage stats or dump this table's contents get weight : report a server's current weight set weight : change a server's weight set server : change a server's state or weight set table [id] : update or create a table entry's data set timeout : change a timeout setting set maxconn : change a maxconn setting set rate-limit : change a rate limiting value disable : put a server or frontend in maintenance mode enable : re-enable a server or frontend which is in maintenance mode shutdown : kill a session or a frontend (eg:to release listening ports) show acl [id] : report available acls or dump an acl's contents get acl : reports the patterns matching a sample for an ACL add acl : add acl entry del acl : delete acl entry clear acl <id> : clear the content of this acl show map [id] : report available maps or dump a map's contents get map : reports the keys and values matching a sample for a map set map : modify map entry add map : add map entry del map : delete map entry clear map <id> : clear the content of this map set ssl <stmt> : set statement for ssl [root@docker_node1 ~]# 提示：從上面的幫助信息，我們可以了解到我們可以通過管道把對應(yīng)指令傳給/var/lib/haproxy/stats文件，從而實現(xiàn)管理后端server的目的； 示例：通過socat命令把web1標(biāo)記為disable
提示：我們執(zhí)行把web1標(biāo)記為disable，系統(tǒng)提示我們沒有權(quán)限；接下來我們來配置權(quán)限即可
提示：以上配置表示該socket文件具有admin權(quán)限，接下來在來把web1標(biāo)記為disable（修改haproxy配置文件，需要重啟haproxy ）
提示：我們修改了權(quán)限以后，現(xiàn)在執(zhí)行以上命令就沒有報什么權(quán)限之類的錯誤了，接下來我們打開狀態(tài)頁，看看web1的狀態(tài)是什么
提示：可以看到web1變成了維護的模式； 示例：列出監(jiān)控頁面指標(biāo)數(shù)據(jù)的信息 [root@docker_node1 ~]# echo "show info " |socat stdio /var/lib/haproxy/stats Name: HAProxy Version: 1.5.18 Release_date: 2016/05/10 Nbproc: 4 Process_num: 3 Pid: 6374 Uptime: 0d 0h05m04s Uptime_sec: 304 Memmax_MB: 0 Ulimit-n: 8039 Maxsock: 8039 Maxconn: 4000 Hard_maxconn: 4000 CurrConns: 0 CumConns: 3 CumReq: 10 MaxSslConns: 0 CurrSslConns: 0 CumSslConns: 0 Maxpipes: 0 PipesUsed: 0 PipesFree: 0 ConnRate: 0 ConnRateLimit: 0 MaxConnRate: 1 SessRate: 0 SessRateLimit: 0 MaxSessRate: 1 SslRate: 0 SslRateLimit: 0 MaxSslRate: 0 SslFrontendKeyRate: 0 SslFrontendMaxKeyRate: 0 SslFrontendSessionReuse_pct: 0 SslBackendKeyRate: 0 SslBackendMaxKeyRate: 0 SslCacheLookups: 0 SslCacheMisses: 0 CompressBpsIn: 0 CompressBpsOut: 0 CompressBpsRateLim: 0 ZlibMemUsage: 0 MaxZlibMemUsage: 0 Tasks: 14 Run_queue: 1 Idle_pct: 100 node: docker_node1 description: [root@docker_node1 ~]# 提示：以上信息就是監(jiān)控頁面中的指標(biāo)數(shù)據(jù)； 示例：上線web1
在監(jiān)控頁面看web1的狀態(tài)
提示：可以看到web1已經(jīng)正常上線了；聰明的你一定想到了通過這樣的方式動態(tài)的去管理后端server；這樣就可以實現(xiàn)很多功能，比如通過zabbix去監(jiān)控haproxy里的狀態(tài)指標(biāo)；通過腳本動態(tài)的上線和下線server、動態(tài)的修改后端server的權(quán)重（這個需要根據(jù)算法來，如果調(diào)度算法是動態(tài)的就支持，靜態(tài)的就不支持）； |
|
|
