Would you like to discuss your environment with a knowledgable engineer?
aiScaler acts as an intermediary between users/visitors on the Internet and your actual (origin) web servers. It is the origin web servers that aiScaler uses to obtain responses from. It is easy to see that aiScaler has to know how to get to the origin servers, for each configured website. The format of origin setting in aicache server config file:
origin <origin ip address> <port> [tag] [weight]
At least one origin server needs to be specified for each accelerated website. The same origin server can be used in different websites – using the same IP and port number or different ones. In some cases, you might choose to specify the same origin server more than once even for a given website, if you want to have aiScaler drive proportionally more requests to it – but consider using weighted load balancing metric instead.
When more than one origin server is specified for a web site and unless configured otherwise, aiScaler uses these origin servers in a round-robin fashion, where each healthy server gets equal share of traffic. In the example above, each origin server will receive one third of the traffic. Other load-balancing metrics are available, including least connections and weighted distribution – we describe those in the next section.
As you know by now, simple round-robin is the default load balancing metric used by aiScaler. When so configured, each origin server gets a share of the traffic proportional to the number of times it is specified as OS for a given website. When each OS is only specified once, each OS gets an equal share of the traffic (1/3 of it, with 3 OS specified).
This is the metric we suggest you use when all of the origin servers (for a given website) are of about the same capacity and configuration and can cope with traffic on equal terms. In some cases, you might choose to specify the same origin server more than once for a given website, if you want to have aiScaler drive proportionally more requests to it – but consider using a weighted load balancing metric instead.
Instead of round-robin load balancing metric, you can configure aiScaler for least connections metric, where the next request is sent to the origin server that is processing the fewest number of requests, at the moment of decision making. This metric has potential to automatically load balance requests in a fair fashion, when origin servers are of different capacity, build and configuration. To configure a given website for least connection load balancing metric, specify
in the respective website’s section. When configured for least connections, number of outstanding connections to each origin server are shown via Web interface and reported via SNMP.
And lastly, you can configure aiScaler for weighted round-robin, when you manually specify the weight for each origin server. The weight is specified as last numeric parameter in the origin configuration line – after server port and server tag. Both server port (even if it is default port of 80) and server tag (even if you want to have default tag of 0) must be specified, if you want to provide weight value – as otherwise aiScaler won’t know which value is port, which is tag and which is weight.
Let’s say you specify 3 origin servers, with weights of 10,20 and 30. It means that out of every 60 requests (10+20+30), the first server will get 10 of them, the second 20 and the third 30. Please note how in the example below we have specified both the port numbers and the OS Tag of ‘0’ for all three Origin Servers! Both server ports (even if it is default port of 80) and server tag (even if you want to have default tag of 0) must be specified, if you want to provide weight value – as otherwise aiScaler won’t know which value is port, which is tag and which is weight.
origin 127.0.0.1 8888 0 10
origin 127.0.0.1 8889 0 20
origin 127.0.0.1 8890 0 30
aiScaler doesn’t enforce any limit on the value of weight, but we suggest to keep it reasonable. aiScaler does its best to smooth out the flow of requests to weighted origin servers, but this approach has its limits when significantly different weights are assigned to origin servers.
To reiterate, to set up weighted load balancing, all you need to do is to specify weights for each origin server. There’s no separate directive you need to provide outside of providing weights.
aiScaler complains loudly and exits when a website is configured for both least connection and weighted load balancing, as it is a fairly nonsensical configuration.
Please note that aiScaler does not support OS tags when configured for least connections or weighted load balancing metrics. To employ OS tagging, you must use default, round-robin load balancing metric.
Some sites resort to having a “disaster-recovery” read-only version of their sites maintained (for example, some sites use periodic, controlled, web-crawling of main sites), so that in case of a catastrophic failure with their main hosting infrastructure, user traffic could be served off such content replica.
To configure such origins of last resort, simply create one or more of origin servers with os_tag of 100. When aiScaler fails to fill cacheable request from regular origin servers and no stale version of cached content is available on aiScaler servers, it will attempt a fill again an origin server of last-resort.
Please note that when website health checks are configured, aiScaler will perform regular health checks against such origin servers of last-resort, so they must be able to properly respond to the health checks, otherwise aiScaler will fail these origin servers just as it would any regular origin server failing a health check.
Detecting a failed LR OS in timely fashion is beneficial, as it allows aiScaler to deal with this condition in a proper fashion, as opposed to trying to direct requests at a bad server.
Origin servers defined with os_tag of 100 are only used as OS of last resort and are not used to deliver any traffic under normal operating conditions.
You might have a need to be able to select origin servers based on request’s URL. For example, you might need to make sure that requests containing login.jsp are only sent to origin servers 220.127.116.11, 18.104.22.168, 22.214.171.124, while requests containing search.jsp need to go to origin servers 126.96.36.199 and 188.8.131.52.
Possible reasons for use of this technique include server partitioning (aka freshly minted sharding) – where different user accounts reside on and are served by, different origin servers and/or database servers. Or you may need to configure all POST requests to go to “write” servers, while read requests are directed against “read” servers. Another example: you can send all users with premium accounts to a one farm of servers, while relegating less fortunate visitors to lesser servers and so on.
Different example might include a situation when you’re adding new functionality to your site, to be served by a new server farm (and may be, completely different framework) and you want to make sure that requests are seamlessly and properly routed out across your existing and new servers.
To get such request-driven-origin-server-selection to work, aiScaler allows to specify origin server tag as an optional pattern attribute. You can also tag your origin servers with origin server tags. Now, when request matches a pattern and that patterns specifies an origin server tag, aiScaler make sure request are only sent to origin servers that have matching tags.
You configure the pattern’s origin server tag via os_tag directive in pattern section of the configuration file. Each pattern can have a single origin server tag defined. For example, we define a simple, 0 TTL pattern containing .AAA and request that matching patterns are sent to origin servers that are tagged with 5:
pattern .AAA simple 0
You configure origin server’s os tag by adding the tag number as last parameter, following origin server’s IP address and port number. For example, here we define 3 different origin servers, all tagged with 5:
origin 184.108.40.206 80 5
origin 220.127.116.11 80 5
origin 18.104.22.168 80 5
Please note that in order to tag origin servers, you must provide origin server port number, even if it is the default HTTP port number of 80. If you don’t, then the tag number can be mistaken for port number and your setup will fail to work. Tag value must be under 254.
The patterns can specify any TTL, including 0 . In other words the responses don’t have to be cacheable. Again, you have full power of patterns, both simple and regexp at your disposal so you’re only limited by your imagination.
Please note that os_tag of 100 has a special meaning – it is used to mark an OS server or servers as servers of “last-resort” . See next section for more information on this feature.
There could be a necessity for configuring both – origin selection driven by URL (os tagging) for specific URLs, leaving other URLs to served from origin servers balanced in round-robin or weighted fasion. In such a case you need to define tagged and untagged origin servers, even if these servers are the same.
See the following example of aiScaler server config:
pattern .AAA simple 0
pattern .BBB simple 0
origin 22.214.171.124 80 1
origin 126.96.36.199 80 2
origin 188.8.131.52 80 0 25
origin 184.108.40.206 80 0 75
Requests containing .AAA will be forwarded to origin 220.127.116.11, .BBB to origin 18.104.22.168
All other requests will be balanced by 1/4 and 3/4 proportions to origin 22.214.171.124 and 126.96.36.199