背景

公司最近Exchange邮件系统来构建高可用邮件系统架构。前端负载均衡就是使用了微软的NLB来实现cas的负载均衡。但是方案实施工程中也发现了一些问题，使用NLB在网络中产生大量广播报，出现丢包严重问题。后改用haproxy替代NLB实现cas的负载均衡，但也这就导致了一个问题，最明显的就是用户通过Haproxy来访问邮件系统后 真正达到邮件系统的地址都是Haproxy的地址，在垃圾邮件过滤的时候就无法实现基于IP的过滤，并且也无法记录IP地址信息。为 了解决这个问题我google了很多方案，可以使用硬负载，硬负载可以实现全透明代理让后端邮件服务器获取到用户的真实IP，还有一种解决方案就是 Haproxy的全透明代理。接下来将会介绍下Haproxy的全透明代理的部署。

方案部署

配置Haproxy

#!/bin/bash

wget 

tar zxvf haproxy-1.4.25.tar.gz

cd haproxy-1.4.25

yum install gcc gcc-c++ autoconf automake -y

make TARGET=linux2628 arch=x86_64 USE_LINUX_TPROXY=1 #重点USE_LINUX_TPROXY=1用于编译支持TPTOXY

make install

mkdir /etc/haproxy

cp examples/haproxy.cfg /etc/haproxy

cp examples/haproxy.init /etc/init.d/haproxy

chmod +x /etc/init.d/haproxy

cp haproxy /usr/sbin/

修改配置文件/etc/haproxy/haproxy.cfg

global

log 127.0.0.1  local0

maxconn 409600

chroot /usr/local/share

#  uid 501

#  gid 501

daemon

nbproc 1

pidfile /usr/local/haproxy/logs/haproxy.pid

#  debug

defaults

log global

maxconn 100000

contimeout 500000

clitimeout 3600000

srvtimeout 3600000

option redispatch

retries 6

frontend mail.domain.com

mode http

bind 0.0.0.0:80

log global

option tcplog

redirect location 

frontend owa_443

mode tcp

bind 0.0.0.0:443

default_backend pool_443

log global

option tcplog

backend  pool_443

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

source 0.0.0.0  usesrc clientip #重点,TPROXY需要加上这行。

server cas01 10.130.170.130:443 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:443 check inter 5000 weight 1 rise 2 fall 3

frontend smtp_25

mode tcp

bind 0.0.0.0:25

default_backend pool_smtp

log global

option tcplog

backend pool_smtp

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

source 0.0.0.0  usesrc clientip #重点,TPROXY需要加上这行。

server cas01 10.130.170.130:25 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:25 check inter 5000 weight 1 rise 2 fall 3

frontend pop_995

mode tcp

bind 0.0.0.0:995

default_backend pool_pop

log global

option tcplog

backend pool_pop

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

server cas01 10.130.170.130:995 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:995 check inter 5000 weight 1 rise 2 fall 3

frontend pop_993

mode tcp

bind 0.0.0.0:993

default_backend pool_993

log global

option tcplog

backend pool_993

balance source

#  option forwardfor

#  option originalto

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

server cas01 10.130.170.130:993 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:993 check inter 5000 weight 1 rise 2 fall 3

frontend pop_135

mode tcp

bind 0.0.0.0:135

default_backend pool_135

log global

option tcplog

backend pool_135

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

server cas01 10.130.170.130:135 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:135 check inter 5000 weight 1 rise 2 fall 3

frontend pop_593

mode tcp

bind 0.0.0.0:593

default_backend pool_593

log global

option tcplog

backend pool_593

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

server cas01 10.130.170.130:593 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:593 check inter 5000 weight 1 rise 2 fall 3

frontend pop_60001

mode tcp

bind 0.0.0.0:60001

default_backend pool_60001

log global

option tcplog

backend pool_60001

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

server cas02 10.130.170.130:60001 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:60001 check inter 5000 weight 1 rise 2 fall 3

frontend pop_55000

mode tcp

bind 0.0.0.0:55000

default_backend pool_55000

log global

option tcplog

backend pool_55000

balance source

option redispatch

option abortonclose

option persist

stick on src

stick-table type ip size 10240k expire 240m

server cas02 10.130.170.130:55000 check inter 5000 weight 1 rise 2 fall 3

server cas02 10.130.170.131:55000 check inter 5000 weight 1 rise 2 fall 3

frontend vs_stats :8081

mode http

log global

option httplog

default_backend stats_backend

backend stats_backend

mode http

stats enable

stats uri /stats

stats auth admin:admin

3.由于ExchangeRPC的端口是动态端口，haproxy必须使用固定端口，我这里改成了60001和55000，修改注册表即可，这里不做介绍。

4.配置TProxy代码如下vi iptables.sh

#!/bin/bash

/sbin/iptables -F

/sbin/iptables -t mangle -N DIVERT

/sbin/iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT

/sbin/iptables -t mangle -A DIVERT -j MARK --set-mark 1

/sbin/iptables -t mangle -A DIVERT -j ACCEPT

/sbin/ip rule add fwmark 1 lookup 100

/sbin/ip route add local 0.0.0.0/0 dev lo table 100

上面的代码目的是为了让所有进入网卡的mangle表的包都打上标记，然后新增一条路由规则将这些打了标记的数据包发送至本地回环接口进行处理。

chmod +x iptables.sh

./iptables.sh

修改配置 /etc/sysctl.conf

net.ipv4.ip_forward = 1

net.ipv4.conf.default.rp_filter = 2

net.ipv4.conf.all.rp_filter = 2

net.ipv4.conf.eth0.rp_filter = 0

net.ipv4.conf.all.send_redirects = 1

net.ipv4.conf.default.send_redirects = 1

7.到此为此基于TProxy的Haproxy全透明代理完成了。为真实实现高可用，haproxy也是单点故障，keepalived也不在这里介绍。

然后将两台cas server的网关改为haproxy的ip。现在可以查看到垃圾邮件的ip。

希望这篇文章可以帮助到你。总之，同学们，你想要的职坐标IT频道都能找到！