Hello Puppet features

I am writing this post as part of a course called Linuxin keskitetty hallinta (Linux centralized management) held by Tero Karvinen http://terokarvinen.com/2013/aikataulu-%E2%80%93-linuxin-keskitetty-hallinta-%E2%80%93-ict4tn011-4-syksylla-2013

I am using 64-bit Xubuntu 13.10 (Saucy Salamander) Fresh install. Everything on this post is done locally on one computer and not over a network to multiple computers.

Note to self: do not use capital letters on classnames. Debugging takes a long time.
Also: --verbose --debug flags are useful when applying with puppet.

The tasks of today’s assignment:

I started by installing puppet:
$ sudo apt-get update && sudo apt-get install puppet

Under the home directory I created “puppet” and “modules” folders
$ mkdir -p puppet/modules

I navigated under puppet and created a folder called “operator” and under that the “templates” folder.
$ cd puppet/
$ mkdir -p modules/operator/templates

I created a new template file
$ nano modules/operator/templates/data.erb

which contained the following:

Hello <%= @id  %>,

here is some information possibly relevant to you if you want to host something online:

Computer: <%=@hostname  %>

ip address: <%=@ipaddress  %>
netmask: <%=@netmask  %>
network interfaces: <%=@interfaces  %>

operating system: <%=@operatingsystem  %> <%=@operatingsystemrelease %>

I created a “manifests” -folder next
$ mkdir modules/operator/manifests

And started to edit init.pp
$ nano modules/operator/manifests/init.pp

class operator {
        file { '/tmp/administrative_data':
            content => template('operator/data.erb'),
        }
}

I applied the changes to puppet
$ puppet apply --modulepath modules/ -e 'class {"operator":}'

Notice: /Stage[main]/Operator/File[/tmp/administrative_data]/ensure: defined content as '{md5}c5767ff1085c775b628c7af88dd270af'
Notice: Finished catalog run in 0.08 seconds

And I checked what it had created:
$ cat /tmp/administrative_data

Hello samuel,

here is some information possibly relevant to you if you want to host something online:

Computer: samuel-School,

ip: 172.28.9.84
netmask: 255.255.0.0
network interfaces: eth0,lo

operating system: Ubuntu 13.10

For the second module I chose Varnish.

I first installed apache2 as in my last post: https://kontsu.wordpress.com/2013/11/01/hello-puppet-module/

after apache2 was installed and tested I decided to also make WordPress working in order to really test Varnish after it has been installed. https://kontsu.wordpress.com/2012/10/10/apache-2-performance-boost-with-varnish-yslow/

After WordPress was installed, I tested it with apache2′s ab.
$ ab -n 1000 http://localhost/~samuel/wordpress/wordpress/

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.4.6
Server Hostname:        localhost
Server Port:            80

Document Path:          /~samuel/wordpress/wordpress/
Document Length:        10072 bytes

Concurrency Level:      1
Time taken for tests:   28.966 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      10344000 bytes
HTML transferred:       10072000 bytes
Requests per second:    34.52 [#/sec] (mean)
Time per request:       28.966 [ms] (mean)
Time per request:       28.966 [ms] (mean, across all concurrent requests)
Transfer rate:          348.73 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    26   29   3.0     27      44
Waiting:       24   26   3.0     25      41
Total:         26   29   3.0     27      44

Percentage of the requests served within a certain time (ms)
  50%     27
  66%     29
  75%     30
  80%     31
  90%     33
  95%     35
  98%     39
  99%     41
 100%     44 (longest request)

I moved on creating a folder for Varnish under puppet
$ mkdir -p modules/varnish/templates

I copied the original apache2’s ports.conf as ports.conf.erb
$ cp /etc/apache2/ports.conf modules/varnish/templates/ports.conf.erb

I changed the line “Listen 80” to “Listen 8080”
$ nano modules/varnish/templates/ports.conf.erb

# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default

Listen 8080

<IfModule ssl_module>
	Listen 443
</IfModule>

<IfModule mod_gnutls.c>
	Listen 443
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

I created a new file called varnish.erb with the following content (which I copied from my earlier post about Varnish)
$ nano modules/varnish/templates/varnish.erb

# Configuration file for varnish
#
# /etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and $MEMLOCK
# to be set from this shell script fragment.
#
# Note: If systemd is installed, this file is obsolete and ignored.  You will
# need to copy /lib/systemd/system/varnish.service to /etc/systemd/system/ and
# edit that file.
 
# Should we start varnishd at boot?  Set to "no" to disable.
START=yes
 
# Maximum number of open files (for ulimit -n)
NFILES=131072
 
# Maximum locked memory size (for ulimit -l)
# Used for locking the shared memory log in memory.  If you increase log size,
# you need to increase this number as well
MEMLOCK=82000
 
# Default varnish instance name is the local nodename.  Can be overridden with
# the -n switch, to have more instances on a single server.
# INSTANCE=$(uname -n)
 
# This file contains 4 alternatives, please use only one.
 
## Alternative 1, Minimal configuration, no VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# content server on localhost:8080.  Use a 1GB fixed-size cache file.
#
# DAEMON_OPTS="-a :6081 \
#              -T localhost:6082 \
#        -b localhost:8080 \
#        -u varnish -g varnish \
#            -S /etc/varnish/secret \
#        -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
 
 
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.  Use a 1GB
# fixed-size cache file.
#
DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"
 
 
## Alternative 3, Advanced configuration
#
# See varnishd(1) for more information.
#
# # Main configuration file. You probably want to change it :)
# VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=
# VARNISH_LISTEN_PORT=6081
#
# # Telnet admin interface listen address and port
# VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
# VARNISH_ADMIN_LISTEN_PORT=6082
#
# # The minimum number of worker threads to start
# VARNISH_MIN_THREADS=1
#
# # The Maximum number of worker threads to start
# VARNISH_MAX_THREADS=1000
#
# # Idle timeout for worker threads
# VARNISH_THREAD_TIMEOUT=120
#
# # Cache file location
# VARNISH_STORAGE_FILE=/var/lib/varnish/$INSTANCE/varnish_storage.bin
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
# VARNISH_STORAGE_SIZE=1G
#
# # File containing administration secret
# VARNISH_SECRET_FILE=/etc/varnish/secret
# 
# # Backend storage specification
# VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
#
# # Default TTL used when the backend does not specify one
# VARNISH_TTL=120
#
# # DAEMON_OPTS is used by the init script.  If you add or remove options, make
# # sure you update this section, too.
# DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
#              -f ${VARNISH_VCL_CONF} \
#              -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
#              -t ${VARNISH_TTL} \
#              -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
#          -S ${VARNISH_SECRET_FILE} \
#              -s ${VARNISH_STORAGE}"
#
 
 
## Alternative 4, Do It Yourself
#
# DAEMON_OPTS=""

I made a new folder for the manifest and created the init.pp with the following content
$ mkdir modules/varnish/manifests
$ nano modules/varnish/manifests/init.pp

class varnish {

	package {"varnish":
		ensure => "installed"
	}

	package { "apache2":
	ensure => "installed"
	}

	file {'/etc/apache2/ports.conf':
		ensure => file,
		require => Package['apache2'],
		content => template('varnish/ports.conf.erb'),
	}

	file {'/etc/default/varnish':
		ensure => file,
		require => Package['varnish'],
		content => template('varnish/varnish.erb'),
	}
}

I applied the changes with puppet and ran the test again
$ sudo puppet apply --modulepath modules/ -e 'class {"varnish":}'

$ ab -n 1000 http://localhost/~samuel/wordpress/wordpress/

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.4.6
Server Hostname:        localhost
Server Port:            80

Document Path:          /~samuel/wordpress/wordpress/
Document Length:        10072 bytes

Concurrency Level:      1
Time taken for tests:   0.405 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      10403989 bytes
HTML transferred:       10072000 bytes
Requests per second:    2467.63 [#/sec] (mean)
Time per request:       0.405 [ms] (mean)
Time per request:       0.405 [ms] (mean, across all concurrent requests)
Transfer rate:          25071.49 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   4.9      0     157
Waiting:        0    0   4.9      0     157
Total:          0    0   4.9      0     157

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%    157 (longest request)

Looks fine, then I happened to run it again and get a score of 2

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.4.6
Server Hostname:        localhost
Server Port:            80

Document Path:          /~samuel/wordpress/wordpress/
Document Length:        10072 bytes

Concurrency Level:      1
Time taken for tests:   0.271 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      10405000 bytes
HTML transferred:       10072000 bytes
Requests per second:    3685.89 [#/sec] (mean)
Time per request:       0.271 [ms] (mean)
Time per request:       0.271 [ms] (mean, across all concurrent requests)
Transfer rate:          37452.80 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.1      0       2
Waiting:        0    0   0.1      0       2
Total:          0    0   0.1      0       2

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      2 (longest request)

After trying different modifications to the configuration files I failed to find the error (if there is any). I also Tested on a fresh xubuntu install by installing the same setup manually (https://kontsu.wordpress.com/2012/10/10/apache-2-performance-boost-with-varnish-yslow/) and it made no difference.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s