http://www.brianhouk.com/papers/f5-bigip.html
brian
at brianhouk
dot com
Hey, yeah you. . . this paper is rather incomplete, which is why if you
have something you would like added fire me an e-mail.
on your bigip echo "hey why don't you add a section about subject <insert
your subject here> | mail brian
at brianhouk
dot com
That or use your e-mail client to send me an e-mail.
The BIGIP from F5 (www.f5.com) can be somewhat pricey, I think
that it's well worth every penny. It's made it easy to do physical network
moves, masking thousands of connections per second for dynamic content across
dozens of servers, and added another layer of redundancey into our network.
The bigip's however can handle much more than the simple tasks which I
have them doing. In load balancing the BIGIP's continually monitor the
servers and other equipment that it monitors for availability. In the event
that one of the devices it manages is down the BIGIP(S) will no longer forward
connections to that device. BIGIP's monitor devices in several different ways
ranging from sending simple ICMP packets to the device to Extended Content Verification(ECV)
which veriifies whether or not a server returns the correct server content.
Upon the first time of booting your BIGIP's you're going to be prompted
with a first time configuration utility. This utility can be called back at
about any time by simply typing "setup". This however have I have
seen will not save you in the event you screw your bigip up royally, it will
not undo previously defined VIPS pools or anything like that. It will not modify
/etc/bid.conf or /etc/bigip.conf (I think) In that utility you will have the
opportunity to configure the web-based configuration utility which is used on
the BIGIP's. I'm not a large fan of this, so I don't use it at all, and
that's all you'll hear about the web based configuration interface from now
on.
Load balancing:
BIGIP's offer several different methods to balance traffic and distribute
requested connections across the servers designated to have those connections
mapped out across. Those are as listed:
ˇ
Round Robin Round Robin is the most basic and most
likely the most commonly used method of load balancing. It distributes connections
evenly across all server ports, passing connections to the next server port
in line. Meaning that if you have 200 connections being sent to the bigip
and 4 servers on the backend recieving those and responding to those connections
that each server on the back end is going to be getting 50 connections. Round
Robin is fine if all the servers that the BIGIP's are mapping connections out
to are the same hardware. In the event that they aren't all the same hardware
you're going to be either putting too high a load on certain machines or too
little of a load on certain machines. Say you node 1 which is capable of handling
90 static connections a second and then you have node 2 which is capable of
handling 200 static connections a second. Well if you're using round robin to
map your connections across to these two servers they're going to be used equally
when node 2 can clearly take more than 200% more connections than node1. So,
by using round robin you're drastically reducing the number of possible connections
that your network can take as well as putting node 1 in the position to get
easily overloaded.
ˇ
Ratio Ratio distributes new connections across server
ports in proportion to a user-defined ratio. This could be used in the
situation which I talked about with the problems in using Round Robin. You could
define for node 1 to take 33 percent of the hits and node 2 to take 66 of the
hits.
ˇ
Priority This method distributes connections in
round robin fashion to a specific group of servers. It begins distributing connections
to the highest priority group. If all servers in that group should go
down, it begins distributing connections to servers in the next higher priority
group. I really haven't ever used this method in load balancing, I just don't
like the sounds of it.
The three previously mentioned load balancing methods all
fall into the category of static load balancing, the next four which will be
explained are dynamic methods of load balancing. Dynamic methods of load balancing
usually use current performance information from each node to decide where to
send the connection to.
ˇ
Least Connections In this mode the BIGIP's check
to see which node has been recieving the least connections and then sends the
connection request to that node.
ˇ
Fastest In this mode the BIGIP's are going to send
the connection request to the node which has the best response time.
ˇ
Observed In this mode BIGIP's send each new connection
to the node that has the highest performance rating based on a combination of
fewest connections and best response time.
ˇ
Predictive This factors in both performance ratings
and performance improvement over time.
It's hard to say which method of load balancing is
going to be the best for your needs. In my situation Ratio is the best for our
needs. Most of the dynamic methods of load balancing just don't provide us with
any benefit. Network connectivity is not going to be a problem with any of the
servers, they'll all have almost the same response time. Now as far as fastest
in dynamic load balancing, we don't want one single server taking a majority
of the connections because it may be the most powerful.
In setting the mode use the command "bigpipe lb <mode>"
with your mode being priority, least_conn, fastest, observed, or predictive.
If you use either ratio or priority mode you're going to have to set
a special property on every node address. This is not done using the bigpipe
lb command. To set ratio weights you're going to use the command bigpipe
ratio nodeIP nodeIP ratio weight. There are two nodeIP's listed in
this example, you don't have to use two, only one can be used or multiple
ones can be used. Here are some examples in setting the weight:
bigpipe ratio 10.3.2.110 10.3.2.111 10.3.2.112 10.3.2.113 10.3.2.114 1
bigpipe ratio 10.3.2.210 10.3.211 2
The Above example would set the first four IP's to distribute 50%
of the connections across the first 4 nodes, and 50% across the last 2 nodes.
In trying to set the priority level of the nodes, the same command would be
used above, but instead of using ratio in the command you would use priority.
bigpipe priority nodeIP nodeIP nodeIP 1 etc. . .
Network and Traffic Control
It never hurts to filter traffic, or have the ability and know-how to apply
traffic filters in the event that you need to. You never know when
you may need to block access from a certain network, a certain network IP,
or access to a certain IP. Hey, let's face it you never know what's
going to go on when dealing with servers on the internet. Aside from
IP filtering you can apply rate filters as well.
man ipfw
IPFW can be use dto do both ipfiltering and rate filtering, watch http://www.brianhouk.com
for a HOWTO on using IPFW.
vips
Defining BIGIP vip's you're going to use the command bigpipe vip.
More information on this command can be found in the bigpipe manpage.
bigpipe vip <virtual address>:<port> [/bitmask] <interface
name> [Unit ID] [netmask] <broadcast IP> define <Node
IP>:<Port> <NodeIP>:<Port> <NodeIP>:<Port>
bigpipe vip 192.168.3.88:80/24 exp0 define 10.3.2.110:80 10.3.2.111:80
10.3.2.112:80 10.3.2.113:80 10.3.2.114:80 would be an example of how
to setup a VIP for 192.168.3.88 on port 80.
Nat & Snat
Mapping external ips to internal ips using either nat or snat are used
in relatively the same manner. Bigpipe snat or bigpipe nat. It will of course matter if you’re
using snat or just regular nat.
Bigpipe nat:
Bigpipe nat <internal Ipaddr> to <external Ipaddr> </bitmask>
<interface name(exp0)>[unit ID]
Bigpipe nat 192.168.3.87 to 10.3.2.101
Bigpipe snat:
Bigpipe snat map <internal ipaddr> to <external Ipaddr> </bitmask>
<interface name(exp0)> [unit ID]
Bigpipe snat map 192.168.3.87 to 10.3.2.101 10.3.2.102 10.3.2.103 10.3.2.104
10.3.2.105 10.3.2.106
Optimizing your BIGIP configuration:
I really don’t have that much experience in this field, so anything
I’m writing really isn’t from personal experience, some of it
is just common sense though. (note,
this is not implying that the author has common sense)
Things which you’ll want to ensure are that your bigip’s have
enough ram in them, this will depend on your situation of course and if
you’re using the ssl accelerator as well. Adding rules to filter traffic
or rate limit traffic are going to take a toll (however small it may be)
in performance on your machine. That falls in the category of common sense/knowledge.
Also to reduce network congestion you’re going to want to reduce
ARP traffic on the external network, and reduce the number of checks done
byt the BIGIP’s. Reducing external arp traffic will help
you more when the BIGIP’s failover. To do this in this manner you’re
going to need to have access to your router, or have your Service Provider
do some configuration on their router for you.
For each virtual address that a BIGIP manages it maintains an IP
alias on the external interface. Those IP aliases are broadcast on that
external network when a virtual server is defined.
Every time that your bigip is activated into active mode, whether
it be from standby or on the initial boot it broadcasts for each of those
virtual addresses. If you have
defined thousands of virtual addresses the ARP requests, all that traffic
sent out onto the network in one burst can throw out quite a bit of congestion. In addition to reducing ARP traffic this
setup will also assist you in failover time. What do you mean “assist
in failover time”? . . The bigip manual says that in extreme circumstances
failover time can take up to 90 seconds, of course this says if the bigip’s
got 8,000 vip’s. To achieve
this you’re going to want to set bigip.vipnoarp command using the
system command sysctl, also you’re going to want to setup on the router
the route to virtual servers using the BIGIP’s external
interface ip. The external interface IP is the ip address shared
by the active and standby BIGIP’s. For the sysctl command sysctl –w
bigip.vipnoarp=1, or edit /etc/sysctl.conf make the correct entry and reboot
the BIGIP. To immediately activate bigip.vipnoarp
mode type the following:
Bigpipe –r Sysctl –w bigip.vipnoarp=1 Bigpipe –f /etc/bigip.conf
Reducing the node pings and service checks done by the BIGIP’s is
something you’re going to want to achieve as mentioned above.
I’m really not going to go into this, but you’re going
to want to look into the bigpipe alias command.
SSL Accelerator
The ssl accelerator helps accelerate SSL
connections in a unique manner. Something which I’ve never seen done
before. It allows incoming
https requests then forwards that request to a regular http server then
forwards that back the the client as https. This allows you to offload SSL processing to the BIGIP’s
instead of letting the webserver take care of it. One problem with this is that this option is intensive on the
bigip’s resources when used often.
In order to use this option you have to have what is called a bigip
accelerator card. The machine
will detect if you have one or not on bootup. This option greatly accelerates
the number of ssl connections which the bigip is cabable of handling. A few steps in Configuring the SSL
Accelerator need to be taken,
they are as follows:
Obtain a Certificate and Generate a Key:
You’ll have to receive a SSL key from some authorized Certification
Authority(CA).
The following are companies which can privide you with that: http://www.verisign.com, http://secure.digitrust.com, http://www.globalsign.com, http://www.cybertrust.gte.com, http://www.entrust.net. If you don’t want to pay for a certificate,
you can generate a test one for yourself to use. You can do so using either
the web-based configuration utility, or you can do it using the ssl utilities,
the easiest way is to use the F5 BIGIP configuration utility.
Incase you haven’t used SSL certificates before, you can not
use the same SSL certificate across multiple domains, IE f5.com and bigip.com
can not use the same SSL certificate. So if you’re planning on using the ssl accelerator on
more than one domain you must have more than one SSL certificate. If you have an old ssl certificate for
a domain, you can use that key, as long as it was made for Apache, OpenSSL,
or Stronghold. (NOTE, BIGIP’s don’t support Microsoft Internet
information Server (IIS/Infected Information Server) Certificates). Anyways
. . . let’s get onto generating your certificate.
If you’re keyless you can generate a key and cert. On the bigip’s
by using the genconf and genkey utilities.
First up run genconf from the root directory cd /; /var/asr/gateway/bin/genconf
This will prompt you for information about the organization that you’re
requesting the certificate for. As standard this will request the following
information from you: Domain name of the server, Two letter ISO code for
your country, Full name of your State / Prov., City or town name which you’re
in, Organization name, and the division name or unit which you’re
a part of. Now that you’ve
run genconf, you have to run genkey, this should be run in the same manner
as running the genekey utility.
Cd /; /var/asr/gateway/bin/genkey domainname.com with domainname.com
the domainname which you’re requesting this cert for. After running this your certificate request form will be stored
in /var/asr/gateway/requests/domainname.com.req. This utility creates a
temporary certificate as well, It will be kept in /var/asr/gateway/certs/domainname.com.cert. Remember, on redundant systems you’re going to have to
copy the key and cert from the primary bigip to the secondary bigip on redundant
systems. The Temporary certificate
is good for only 30 days. If
you do not have a valid certificate by the time that certificate expires(30
days) you will have to redo the steps you have to generate the temporary
certificate. If you already have an existing key and
you want to generate a certificate copy the existing key for the server
into /var/asr/gateway/private. Afterwards
run cd /; /var/asr/gateway/bin/gencert domain.com. Then you will be prompted for some more
information, then a cert. Request will be located in /var/asr/gateway/requests/domain.com.req.
After you have received the certificate from the Certificate Authority
for the SSL Accelerator you must copy it onto both of your BIGIP’s.
Copy the certificate into /var/asr/gateway/certs on both BIGIP’s.
Next up you’re going to have to Create a HTTP Virtual Server.
This most likely isn’t anything new to you, bigpipe vip
192.168.3.88:80 use pool SSLSERVERS This would setup a VIP for all connections
coming in on 192.168.3.88 to port 80 to goto any server in the pool “SSLSERVERS”.
After the creation of the HTTP virtual server, you’ve got to setup
the SSL Gateway. The syntax
for creating a ssl gateway is the following: bigpipe proxy <IPAddress>:<Port
Num> [/bitmask] [<Interface Name(exp0)] [<unit id>] target <server
| vip> ip:port ssl enable key <key> cert <cert>. . more
of this can be found in the manpage for bigpipe.
A real world usage of this would look like: bigpipe proxy 192.168.3.88:443
exp0 unit 1 { netmask 255.255.255.0 broadcast 192.168.3.255 target vip 10.3.2.110:80
ssl enable key domain.com.key cert domain.com.cert )
Enabling, Disabling, or Deleting a SSL Gateway.
This is most likely the easiest thing which you’ll have to
do. Bigpipe proxy <IPAddr>:<Port>
FUNC where FUNC is either going to be enable disable or delete. For example bigpipe proxy 192.168.3.88:443 enable would
turn on the proxy for the HTTPS port on that IP.
Viewing traffic and usage information for the SSL proxy is done as follows.
If you wanted to see it for the proxy which we just enabled you’d
issue the command bigpipe proxy 192.168.3.88:443 show
Advanced Server Check Options:
Advanced Server Checking options are used to verify that your servers are
serving the correct content properly.
There are Two types of service checks, Extended Content Verification
(ECV) and Extended Application Verification(EAV).
Let’s take a look at how to setup these types of checks.
ECV: (for transparent nodes) To check if a node is functioning correctly
you’re going to want to make sure that it’s got an entry in
/etc/bigd.conf. The following is an example of the syntax
used in bigd.conf.
Transparent <node ip>:<node port> <url> [*expression
to receive*]
In this your URL could be in the form of an IP:PORT/path/to/file.html
Here’s an example line for bigd.conf:
Transparent 192.168.3.82:80 192.168.3.84:8080/www/form/check.html *Server
Fine*
This would connect from 192.168.3.82 from port 80 to 192.168.3.84 on port
8080 and fetch the url /www/form/check.html, then it will check the return
for the text *Server Fine*
For ECV checks you’re going to have to start off the line with the
word “transparent” the
node IP can either be NodeIP or a domain name, if you’re going to
use the domain name you need to make sure that the BIGIP is configured for
Name Resolution. Node port is the node port to use for
the ECV check. You can use
0, a numeric port entry, or a well-known service name such as http or https.
DestIP:port can be either the IP or full domain name of the IP you want
to check, followed by the Port or a popular service name. URL is the URL to fetch, the request can not be longer than
4096. The only part of this which is optional is the *Expression To recieve*.
This entry must be enclosed in quotation marks, and can not be longer
than 128 bytes.
When you have finished editing /etc/bigd.conf you’re going to have
to restart bigd.
EAV(Extended Application Verification):
This is a more sophisticated type of service check typically used to confirm
whether an application running on a node is responding correctly to client
requests or not. To determine
this the BIGIP uses a custom program called and external service checker.
This external program allows to customize service check functionality.
This program usually doesn’t come with the BIGIP’s, it’s
usually a program which is written by the customer themselves. The bigip’s
however do ship with a few External Service Check programs, including: ftp,
pop3, smtp, nntp, and sql. These also can be used as examples or refrences
in writing your own external service checking programs.
Being that you can write your own external service checking programs,
there’s really no end to what you could do with this.
In writing your own external service checker program you’re
going to have to have it write output in the event that it’s successful
or not write anything in the event that it’s unsuccessful. There is
no “node up” or “node down” it’s simply, output
if up or nothing if down. Your
program must also create a PID file. This is to hold it’s process ID so the machine knows
what it is. It should be kept
in /var/run/pinger.<IP>..<PORT>.pid The checker should also
make it so that as soon as it starts if a pid file for that program already
exists that it sends the sigkill signal to that program.
It must then write its process number to that pid file once it’s
sure that another one isn’t running.
Also, the external service checker must delete its pid file before
exiting. Example scripts can be found in /usr/local/lib/pingers/*
Here’s Sample_pinger:
#these arguments supplied automatically for all external pingers:
# $1 = IP(nnn.nnn.nnn.nnn notation or hostname)
# $2 – port (decimal, host byte order)
# $3 and higher = addition arguments
#
# In this sample script $3 is the regular expression.
Pidfile=”/var/run/pinger/.$1..$2.pid”
If [ -f $pidfile ]
Then
Kill –9 `cat $pidfile`> /dev/null 2>&1
Fi
Echo “$$” > $pidfile
Echo”GET /” | /usr/local/lib/pingers/nc $1 $2 2> /dev/null | grep –E –I $3 > /dev/null
Status=$?
If [ $status –eq 0 ]
Then
Echo “up”
Fi
Rm –f $pidfile
As you can see scripting is shell scripting, so anything that you can automate
via shell scripting you can monitor with Service Check.
After you have written your script put it in /usr/local/lib/pingers/.
This is the default location for External Service Checker application.
They can also be installed into other directories if you want them
to be. Now that you’ve written and tested
your external service checking program, and you have it installed on the
bigip you have to make the correct entry in /etc/bigd.conf. The format is :
External [<nodeIP>:]<port> [<path>]<pinger_name>
[*<argument_string*>]
<path> is a absolute or relative path to the External checker application.
If it’s an absolute path it should start with a /. The pinger_name is the name of the script on the system. The
argument_string are the arguments, it may only contain one string in quotation
marks. Here’s an example
line for bigd.conf:
external 10.3.2.2:80 /scripts/mysql/mysqlcheck-node1.sh “-d –b”
Now that you have added that to the bigd.conf it will be checked at specified
intervals. It uses the interval
set using the bigpipe tping_svc command.
Eav For:
FTP: external 192.168.3.88:21 /usr/local/lib/pingers/FTP_pinger
“/pub/ls-lR.gz anonyhmous user@domain.tld”
this will attempt to download /pub/ls-lR.gz to the /var/tmp/ directory.
POP3: external 192.168.3.88:109 /usr/local/lib/pingers/POP3_pinger
“bhouk bad_password”
SMTP: external 192.168.3.88:25 /usr/local/lib/pingers/SMTP_pinger bigtest@domain.tld
NNTP: external 192.168.3.88:119
/usr/local/lib/pingers/NNTP_pinger “alt.unix.horrors”
SQL: external 192.168.3.88:3306 “/usr/local/lib/pingers/SQL_pinger”
“database database_user database_pass”
Options for Advanced Persistance:
Aside from simple persistence and SSL persistence the BIGIP offers some
other mehods of persistence. The
other “advanced” methods of persistence will be described bwlow.
HTTP Cookie Persistance:
This method uses a http cookie stored on a client’s computer to determine
which node this client previously dealt with, this is done in order to connect
it with the correct node which it previously used for numerous reasons. There are four types of cookie persistence,
Insert Mode, Rewrite Mode, Passive Mode, Hash Mode. Whichever mode you choose will determine
how it’s handled by the BIGIP when it’s returned to the client.
Insert Mode:
In insert mode the information about the server to which the client is
being connected to is inserted into the header of the HTTP response as a
cookie. The cookie’s
named BIGipServer<POOL_NAME>. It includes the address and port of
the server handling the connection.
The expiration date is set in this cookie based on the timeout which
is set on the BIGIP. To activate
Insert HTTP cookie persistence from the command line use the following command:
bigpipe pool <pool name> { <lb_mode_specification> persist_mode
cookie cookie_mode insert cookie_expiration <timeout> <member definition>
}
The <timeout> value is written using the format <days>d hh:mm:ss
ReWrite Mode:
If you’re using rewrite mode, the BIGIP intercepts a SetCookie named
BIGipCookie sent from the server to the client, it overwrites the name and
value of the cookie. The cookie
is re-written to be named BIGipServer<pool_name> and includes the
address and port of the server handling the connection. Rewrite requires you to setup the cookie
created by the server. To work
correctly there needs to be a blank cookie coming from the server which
the bigip will re-write. With
apache the cookie can be added to every web page header by adding an entry
in the httpd.conf file like the following:
Header add Set-Cookie
BIGipCookie=000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
That is 120 Zero’s. To activate ReWrite Mode cookie persistence
issue the following command:
Bigpipe pool <pool_name> { <lb_mode_specification> persist_mode
cookie cookie_mode rewrite cookie_expiration <timeout> <member
definition> }
And just like before. The <timeout> value is written using the format
<days>d hh:mm:ss.
Passive Mode:
If you use passive mode the BIGIP doesn’t insert or search for blank
Set-Cookies in the response from the server. It doesn’t try to setup the cookie
either, it assumes that the server provides the cookie formatted with the
correct node information and timeout.
So the webserver needs to send a cookie with the correct information,
because the BIGIP’s not going to touch it. This is easy to setup in
apache, by adding a line to the httpd.conf every page that the server sends
out will have the cookie added to it.
The line for httpd.conf is:
Header add Set-Cookie: “BIGipServer my_pool=184658624.20480.000;
expires=Sat, 19-Aug-2000 19:35:45 GMT; path=/”
In the following example my_pool is the name of the pool which has that
server node in it.
184658624 is the encoded node address and 20480 is the encoded port number.
The node address is achieved with the formula d*256^3+c*256^2+b*256+a, and
the formula to get the port number encoded is to take the two bytes that
store the port and reverse them. So 80 = 80 * 256 + 0 = 20480.
Activating Passive mode is achieved with the command:
bigpipe pool <pool_name> <pool_name> { <lb_mode_specification>
persist_mode cookie cookie_mode passive <member definition> }
Nope, no timeout mode. J
Hash Mode:
Hash mode consistently maps a cookie value to a specific node.
When that client returns the bigip uses the cookie information to
return the client to a given node. With this setup the web server must generate
the cookie. The bigip doesn’t
create the cookie automatically like it does in insert mode. The following command can be used to enable
hash mode:
Bigpipe pool <pool_name> { <lb_mode_specification> persist_mode
cookie cookie_mode hash cookie_hash_name <cookie_name> cookie_hash_offset
<cookie_value_offset> cookie_hash_length <cookie_value_length>
<member_definition> }
Cookie_name is the name of the cookie set by a website.
Cookie_value_offset is the number of bytes in the cookie to skip before
calculating the hash value.
Cookie_value_length is the number of bytes to use when calculating the
hash value.
Advanced Redundant System Features:
In addition to the basic redundancy features of the BIGIP’s there
are more features which will help ensure that your content is available
for those which request it. Some of the following are such features: Mirroring
Connection and Persistence information, Gateway Fail-Safe, Network-based
Fail-over, Setting a specific BIGIP to be the active controller, Setting
up an active active redundant controller.
Mirroring Connection and Persistance Information:
When your primary active controller fails over and the BIGIP which was
standby becomes active if not
configured correctly would drop all real-time connections or persistence
information state. To avoid
loosing that information in the event the primary bigip fails mirroring
of that information has to be enabled.
That way when the failover occurs it can be seam-less and file transfers can continue uninterrupted.
Both clients don’t see any change, and the servers don’t see
a change either. The commands
you’re going to use are the following: bigpipe mirror, bigpipe
vip mirror, bigpipe snat mirror.
So to enable global mirroring execute the command bigpipe mirror
enable, other flags to that command other than enable, are disable,
and show.
Mirroring Virtual Server State:
Mirroring allows for seamless failover from the active bigip to the standby
bigip. It allows mirring of
the persistence information, SSL persistence, or sticky persistence. Mirroring is not meant to be used on all
connections, so it is not enabled by default and you must enable it for
whichever vip’s you want it to be used on. The command vip <VIRT ADDR>:<PORT> mirror
[ persist | conn] enable | disable
Persist is used to monitor persistence information for the Virtual
Server. Conn is used to mirror connection information for the virtual server.
The following command would be used to display connection information
bigpipe <VIRT ADDR>:<PORT> mirror [ persist | conn ] show. If you do not specify either conn or persist
then both will be displayed.
Mirroring SNAT Connections:
SNAT connections are not mirrored by default. You can enable specific SNAT connection
mirroring by a specific node
address and by enabling mirroring for the default SNAT address. The following command can be used to enable
SNAT connection mirroring on an address: bigpipe snat <NODE ADDR>
[<NODE ADDR>] mirror enable | disable Example: bigpipe snat
192.168.3.88 mirror enable, and the example to turn it on for the default:
bigpipe snat default mirror enable | disable.
Gateway Fail-Safe:
This uses features on the BIGIP to provide network failure detection based
on network traffic. Gateway failsafe monitors traffic between the BIGIP
and the gateway router. This
protecting the system from a loss of internet connectivity by triggering
a failover when the gateway is unreachable. In order to enable gateway fail-safe you’ve
got to change the settings of three BIG/db keys using the bigdba utility.
First type bigdba to open the BIG/db database.
To set the IP of the router type the follwing: Local.Bigip.GatewayPinger.Ipaddr=ip.add.re.ss To set the ping interval type the following:
Local.Bigip.GatewayPinger.Pinginterval=20 where the number at the
end is the number of seconds which you want the BIGIP to wait before pinging
the router. To set the timeout
which the BIGIP waits before marking the gateway down type: Local.Bigip.GatewayPinger.Timeout=20
after setting these variables, to close and save your changes type quit
and hit enter. Bigd should
be restarted after this has been done.
Now enable gateway failsafe by issuing the command bigpipe gateway
failsafe arm and then to
turn it off type bigpipe gateway failsafe disarm. To see the current status for gateway fail-safe issue the command
bigpipe gateway failsafe show.
Network Based Failover:
This allows you to configure your redundant BIGIP’s to use the network
to determine the status of the active controller. Network based failover
can be used in replacing hard-wired failover, or it can be used in addition
to hard-wired failover. Configuring
Network based failover is done in somewhat the same way which Gateway fail-safe
was done, using the bigdba utility. So type bigdba then type
Common.Sys.Failover.Network=1 after entering that command type quit
and you’re done in bigdba. Don’t
forget that you’re going to have to restart bigd after doing this.
Setting One BIGIP Active and one BIGIP Standby:
You can do this using the sod command. To force the BIGIP to master type sod
–force_master and to forced the BIGIP to standby type sod –force_slave
. You can type that on the command line, but you should edit the /etc/rc.local
(I belive it is) on both machines and set this command correctly in those
along with the other arguments passed to sod.
Active Active Controllers:
You can have both of the controllers running active at the same time while
still having some failover abilities. Performance also improves when both BIGIP’s
are in active mode at the same time. In active-active you configure the Virtual Servers to be handled
by one of the bigip’s. While
both bigip’s are handling virtual servers, if one of the two BIGIP’s
fail the other will take over the responsibility of the failed BIGIP. In configuring for Active Active mode,
issue the command bigdba. In
bigdba make sure that the Local.Bigip.Failover.UnitId is set to 1 on one
BIGIP, and set to 2 on the other BIGIP. Then make sure on both BIGIP’s
that Common.Bigip.Failover.ActiveMode to 1.
Then define your virtual servers, update the fail-over daemon sod,
and make sure you’ve restarted Bigd.
After you’ve done that it’s now time to configure another
shared IP alias. When you first configured your redundant
BIGIP systems, you setup a shared ip alias which the active controller uses.
In the active active setup you’re going to have to assign another
shared IP alias, that way both Both BIGIP’s have their own shared
IP alias which they each hold. This
is the address behind the BIGIP’s to which the servers behind the
BIGIP’s route traffic. You’ll use the bigip ipalias command
to add another alias. An example would be: bigpipe ipalias exp1 10.3.2.2
netmaks 255.255.255.0 unit 2 If you don’t have a ip alias for
unit 1 add one for unit1 as well. Use bigpipe interface to
view ip aliases. With this
configuration if a BIGIP fails over the remaining BIGIP will assume the
SharedIP address that it had,a dn the servers will continue routing through
the same IP address with nothing changed.
You can also configure additional shared IP aliases on an external
interface. This makes it possible for the routers to route to a virtual
server using vip noarp mode. When
using active-active mode and you have to define virtual servers, nat, or
snat you’re going to have to define which unit it’s going on
[1 or 2]. Examples:
Bigpipe vip 192.168.3.88:80 define unit 1 10.3.2.100:80 10.3.2.101:80 10.3.2.102:80
Bigpipe nat 10.3.2.101 to 192.168.3.87 unit 1
. . . use the unit 1 or 2 flag to set which unit this is going to be on,
that way both units know when one fails which the other has to take over.
. understood? Good. . now we need to update the changes via bigpipe failover init.
Next up synchronize the configuration bigpipe configsync all
Next up login to the active BIGIP and issue the command bigpipe failover
standby, then login to the second BIGIP and issue the same command.
When running in active active mode you have both BIGIP’s fielding
connections for virtual servers. When BIGIP 1 fails BIGIP2 takes over all
of the functions of BIGIP 1, the same goes for when BIGIP2 fails. When one of the BIGIP’s fails in
active active mode I don’t want that second BIGIP coming back up as
active so I have disabled autmatic fail back.
This just so I can find out why the BIGIP failed, and if there are
any problems stop them then and there. To disable automatic failback type
bigdba and then Common.Bigip.Failover.ManFailBack=1. This way it will not failback until the
only active unit is given the command bigpipe failover failback.
If at any time you want to take one of the active active BIGIP’s
out of service just issue the command bigpipe failover standby. This
causes the BIGIP to surrender its unit number to its peer.
Commands To Know:
Bigpipe conn dump
Bigpipe vip persist dump
Bigpipe sticky dump
Bigpipe snat show
Bigpipe unit show
Bigpipe unit peer show
Monitoring & Stuff
Monitoring and administration utilities which come with the BIGIP’s.
bigpipe (man bigpipe) you can issue commands such as bigpipe node
and bigpipe vip. Bigpipe show vip 192.168.3.88 will show
the VIP for 192.168.3.88. bigstat this is used for statistical monitoring.
It allows you to customize display of statistical information. Bigtop¸
this program is like unix’s top but for the BIGIP. Bigtop is beautiful
in giving a non-tech something tech-looking to make them feel all good and
warm. Bigdba is a command which allows
you to modify various configurations for the BIGIP’s. sysctl can also be used in modifying system variables.
As far as console configuration and monitoring goes bigpipe is going to
be your most used utility. I’d recommend getting a printout of the
manpage and keeping it at your desk if you’re going to be doing any
amount of work on the bigip’s.
You can reset stats for any VIP using BIGPIPE VIA: bigpipe vip
ip.add.re.ss stats reset. If you want to reset them for all virtual
servers use the following command bigpipe vip stats reset.
My favorite for all around monitoring and resetting the stats bigpipe
global stats reset. If
you want to monitor the status of all nodes issue the command bigpipe
nodes. It will show if it’s up down disabled
or unchecked. If you want to view stats on one node issue bigpipe node
<node addr> show. To
enable or disable a VIP type the command bigpipe vip <virtual addr>
enable, or you could use disable. You can guess what each command
does. Aside from the
IP you could as well specify the PORT if you just wanted to disable that
PORT vip. You can enable or disable a node in the
same manner, bigpipe node 10.3.2.101 disable, or bigpipe node 10.3.2.101:80
disable
Other Help Resources:
http://tech.f5.com Unfortunately like any other help
that you need with your F5 products you're going to have to be a paying
customer.
brian at brianhouk dot com: I am more than happy to help out with
your problems, It may take me some time and I may just refer
you to the correct documentation or manpage to read or which section.
Located on the bigip filesystem itself there are documents in
pdf format which are almost identical to the bigip manuals.
If you have any questions or find any information contained in
this document which is incorrect I would appreciate an e-mail. brian at brianhouk
dot com.