Solr Replication, Load Balancing, haproxy and Drupal

I use Apache Solr for search on several projects, including a few using Drupal. Solr has built in support for replication and load balancing, unfortunately the load balancing is done on the client side and works best when using a persistent connection, which doesn't make a lot of sense for php based webapps. In the case of Drupal, there has been a long discussion on a patch in the issue queue to enable Solr's native load balancing, but things seem to have stalled.

In one instance I have Solr replicating from the master to a slave, with the plan to add additional slaves if the load justifies it. In order to get Drupal to write to the master and read from either node I needed a proxy or load balancer. In my case the best lightweight http load balancer that would easily run on the web heads was haproxy. I could have run varnish in front of solr and had it do the load balancing but that seemed like overkill at this stage.

Now when an update request hits haproxy it directs it to the master, but for reads it balances the requests between the 2 nodes. To get this setup running on ubuntu 9.10 with haproxy 1.3.18, I used the following /etc/haproxy/haproxy.cfg on each of the web heads:

global
    log 127.0.0.1   local0
    log 127.0.0.1   local1 notice
    maxconn 4096
    nbproc 4
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    maxconn 2000
    balance roundrobin
    stats enable
    stats uri /haproxy?stats

frontend solr_lb
    bind localhost:8080
    acl master_methods method POST DELETE PUT
    use_backend master_backend if master_methods
    default_backend read_backends

backend master_backend
    server solr-a 192.168.201.161:8080 weight 1 maxconn 512 check

backend slave_backend
    server solr-b 192.168.201.162:8080 weight 1 maxconn 512 check

backend read_backends
    server solr-a 192.168.201.161:8080 weight 1 maxconn 512 check
    server solr-b 192.168.201.162:8080 weight 1 maxconn 512 check

To ensure the configuration is working properly run

wget http://localhost:8080/solr -O -
on each of the web heads. If you get a connection refused message haproxy may not be running. If you get a 503 error make sure solr/jetty/tomcat is running on the solr nodes. If you get some html output which mentions Solr, then it should be working properly.

For Drupal's apachesolr module to use this configuration, simply set the hostname to localhost and the port to 8080 in the module configuration page. Rebuild your search index and you should be right to go.

If you had a lot of index updates then you could consider making the master write only and having 2 read only slaves, just change the IP addresses to point to the right hosts.

For more information on Solr replication refer to the Solr wiki, for more information on configuring haproxy refer to the manual. Thanks to Joe William and his blog post on load balancing couchdb using haproxy which helped me get the configuration I needed after I decided what I wanted.

Solr Replication, Load Balancing, haproxy and Drupal

Phine wrote:

Recently, I came across an interesting article which shared deep insights on built in concept of replication in Solr.U can refer to http://www.lucidimagination.com/blog/2009/05/31/solr-index-replication/ for detailed information.

Added Wed, 2010-03-31 23:51

Need to Clarify haproxy with multiple solr servers

Ashok wrote:

hi,

Please let us know how can configure haproxy with multiple solr servers and also let me know how can verify data are going to both servers.

Thanks

Added Wed, 2013-07-17 17:21

security problem ahead

Glenn Plas wrote:

Be aware that anyone who can navigate to your solr core dashboard can actually wreck havoc and drop a core.

Added Tue, 2014-10-07 00:21

RE: security problem ahead

Dave wrote:

@Glenn the security issue you've identified is a problem for all Solr instances regardless of the use of haproxy. Various options are available for restricting access to the dashboard including jetty/tomcat/haproxy config or iptables.

Added Tue, 2014-10-07 05:49

viagra super force 100mg 100mg pills

bapuhemio wrote:

what does female viagra feel like. edhelpprof.mdhelpserv.com - shelf life of viagra tablet,

Added Wed, 2017-12-13 22:02

can i take 75 mg of viagra

chidamizox wrote:

what happens when women take viagra. edhelpnorb.mdhelpserv.com - viagra gel for women,

Added Fri, 2017-12-15 05:58

can i take 75 mg of viagra

chidamizox wrote:

what happens when women take viagra. edhelpnorb.mdhelpserv.com - viagra gel for women,

Added Fri, 2017-12-15 05:58

how long does viagra take to wear off

sabuzahez wrote:

viagra details in hindi. edhelpgamb.mdhelpserv.com - 100mg viagra street value,

Added Sat, 2017-12-16 23:51

should you use viagra if you don't need it

donirabab wrote:

is viagra available in medical stores. edhelpmon.mdhelpserv.com - should you drink when taking viagra,

Added Mon, 2017-12-18 00:31

will viagra affect blood pressure

nniwatea wrote:

how much time does viagra take to act. edhelpfi.mdhelpserv.com - side effects of viagra nhs,

Added Tue, 2017-12-19 09:03

accidentally took 2 viagra

popagetoj wrote:

viagra 100 mg 30'lu tablet. edhelpac.mdhelpserv.com - what is the main ingredient in herbal viagra,

Added Tue, 2017-12-19 15:22

who is the brunette actress in the latest viagra commercial

bamishireg wrote:

viagra psychological side effects. edhelpscin.mdhelpserv.com - viagra to control blood pressure,

Added Wed, 2017-12-20 04:24

can viagra cause dry mouth

ragibiton wrote:

herbal pills that work like viagra. edhelpwe.mdhelpserv.com - can you drink coffee with viagra,

Added Fri, 2017-12-22 11:19

can viagra cause dry mouth

ragibiton wrote:

herbal pills that work like viagra. edhelpwe.mdhelpserv.com - can you drink coffee with viagra,

Added Fri, 2017-12-22 11:20

common doses of viagra

nezeniih wrote:

generic viagra pills sold in nc. edhelpgamb.mdhelpserv.com - alternate between viagra and cialis,

Added Sat, 2017-12-23 12:28

does viagra make men last longer

boerihot wrote:

why do you take viagra on an empty stomach. edhelpmon.mdhelpserv.com - how to get viagra legally in uk,

Added Sat, 2017-12-23 23:58

what does viagra looks like

morumawaz wrote:

how to take viagra alcohol. edhelpwobb.mdhelpserv.com - viagra tablets use in tamil,

Added Sun, 2017-12-24 05:59

how soon can i take viagra after a meal

kokotaed wrote:

chinese alternative to viagra. edhelpga.mdhelpserv.com - prescription for viagra nz,

Added Tue, 2017-12-26 02:32

side effects from viagra use

risetonad wrote:

what does natural viagra do. edhelpnorb.mdhelpserv.com - is it best to take viagra before or after a meal,

Added Wed, 2017-12-27 00:53

do we need prescription viagra

dorezekoz wrote:

how to make your own natural viagra. edhelpscin.mdhelpserv.com - can i take half viagra pill,

Added Wed, 2017-12-27 12:59

can you take viagra after brain surgery

rohawoneh wrote:

brunette model in new viagra commercial. edhelpaz.mdhelpserv.com - 25 mg viagra didnt work,

Added Wed, 2017-12-27 18:13

how do you make natural viagra

hedabekit wrote:

best generic viagra site. edhelpnick.mdhelpserv.com - viagra eyesight side effects,

Added Fri, 2017-12-29 00:37

what is the difference between generic viagra

rudasodzus wrote:

can you make your own viagra. edhelppet.mdhelpserv.com - how soon before intercourse should i take viagra,

Added Fri, 2017-12-29 12:38

hearing loss viagra studies

sakutoroc wrote:

take levitra and viagra together. edhelprei.mdhelpserv.com - should i take viagra on an empty stomach,

Added Sat, 2017-12-30 17:48

stack daily cialis with viagra

edapikil wrote:

how much is viagra 100mg at walmart. edhelpdesk.mdhelpserv.com - where do you get viagra uk,

Added Sun, 2017-12-31 23:23

what do you feel after taking viagra

royupitod wrote:

what happens when woman takes viagra. edhelpun.mdhelpserv.com - less side effects viagra cialis,

Added Mon, 2018-01-01 05:20

how to make viagra from watermelon

nipibens wrote:

do viagra pills lose potency over time. edhelpgamb.mdhelpserv.com - can viagra and levitra be taken together,

Added Mon, 2018-01-01 11:15

how do you know if a guy takes viagra

nitehegod wrote:

viagra for pulmonary edema. edhelpking.mdhelpserv.com - taking cialis viagra together,

Added Fri, 2018-01-05 07:00

cialis and viagra dosage

wosonekus wrote:

when does viagra become generic. edhelpac.mdhelpserv.com - what generic viagra works,

Added Sun, 2018-01-07 22:24

what does female viagra do for women

pumatobeq wrote:

is generic viagra as good as the real viagra. edhelprei.mdhelpserv.com - taking 100 mg viagra reviews,

Added Tue, 2018-01-09 18:50

how to make natural viagra dailymotion

gigoyopeh wrote:

combination of cialis and viagra. edhelpreau.mdhelpserv.com - side effects of too many viagra,

Added Fri, 2018-01-12 05:47

difference between levitra viagra cialis

denashimaj wrote:

how to tell if your partner is taking viagra. edhelpla.mdhelpserv.com - is there a generic viagra yet,

Added Mon, 2018-01-15 21:05

pay with paypal for viagra

shitamusag wrote:

who is the actress in the latest viagra commercial. edhelpbrit.mdhelpserv.com - viagra need prescription uk,

Added Tue, 2018-01-16 10:08

what if you give a woman viagra

bukiisow wrote:

how long does viagra take to expire. edhelpdesk.mdhelpserv.com - what will happen if a woman took viagra,

Added Wed, 2018-01-17 09:32

viagra and cialis made in the usa

nuwogitav wrote:

pfizer viagra china trademark. edhelpli.mdhelpserv.com - safety of viagra from india,

Added Fri, 2018-01-19 15:23

generic viagra pill picture

kigokewam wrote:

how long before viagra should be taken. edhelpnorb.mdhelpserv.com - viagra model in blue dress name,

Added Sat, 2018-01-20 19:27

generic viagra pill picture

kigokewam wrote:

how long before viagra should be taken. edhelpnorb.mdhelpserv.com - viagra model in blue dress name,

Added Sat, 2018-01-20 19:28

cialis vs viagra which is best

hogiwamum wrote:

who is the girl in viagra commercials. edhelpen.mdhelpserv.com - what the best dose of viagra,

Added Sun, 2018-01-21 00:51

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <p> <div> <blockquote> <pre>

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.