As I’ve shared in my previous post, I was working on hardening AWS Linux 2014.09 using Puppet. I got that finished and I’ll probably improve the class in the future to make it easier to manage.
The first thing you’ll realize when you harden a system is this: it will break stuff… No, I’m not talking about the small stuffs, I’m talking about major applications that you rely on.
The first to break that I noticed was running a Puppet master with httpd + mod_passenger in AWS Linux 2014.09. It won’t work properly anymore… Ugh!
Since I was in hurry when I first discovered this, I just reverted back to the default Puppet master using Ruby webrick. Performance was really not an issue since this is only for our environment where we write/test our Puppet classes…
But when I decided to separate the CA server of our Puppet master for scalability — I can’t use the default Ruby webrick anymore. I have to allot the time and investigate if what’s the root cause of the problem
snapshot of the error
Clearly… it has something to do with permissions. The web server (httpd) cannot access the Unix socket created by mod_passenger.
Reviewing the CIS guidelines for AWS Linux 2014.09 pointed me to “3.1 Set Daemon umask” which states:
This is enforced by this Puppet class via this sysconfig configuration. CIS 3.1 guideline clearly says that “The daemon process can manually override these settings if these files need additional permission.” — which gave me an idea to override this in the httpd level.
So… override we go…
I opened /etc/init.d/httpd in vim and added a less strict umask: umask 0022
Restarted httpd and then… FIXED! :)
I encountered this error again when running puppet version 3.8.2 … I know I encountered this error before but I it took me a few minutes to remember how I solved it… so that I won’t forget in the future, I’m posting this here.
The error looks like this:
To solve it, just run: yum install rubygem18-json.x86_64
Somebody forgot to include json gem as a dependency… :)
Update [2015-07-07]: Puppet module is practically done for hardening AWS Linux 2014.09, you can check it out here: https://github.com/proletaryo/cis-puppet
It’s been almost a year since I posted here. Work is very challenging nowadays…
The latest project that I’m part of is now dealing with financial services. Yup, this means a lot of security exercises that need to be done to comply with PCI-DSS (Payment Card Industry Data Security Standards). I find these exercises challenging, a new lens that let’s you understand a lot of things and even makes you paranoid sometimes. IT Security is core – I learned a lot in this area for the past few months.
Anyway, right now I’m working with OS hardening based on the benchmark provided by Center for Internet Security. They provide guidelines on how to do this. Just download the document for your OS here: https://benchmarks.cisecurity.org/downloads/multiform/index.cfm
I’m working mostly in AWS nowadays – It’s a good thing that CIS released a benchmark for AWS Linux 2014.09 version.
We’re a Puppet shop so the first thing I did was to check if there are modules for AWS Linux. the closest one that I’ve found is for RHEL: https://github.com/arildjensen/cis-puppet
Close but not close enough… but definitely better than nothing :)
The beauty of OSS is you can always fork a project and Github is a wonder-tool! So fork I went… I’m already done with CIS Scored guidelines 1.x.x to 3.x.x — a few more to go. Once done, I’m hoping that I can merge this back to master if the original author will allow :)
If you’re interested in this project, just drop me a message here: https://github.com/proletaryo/cis-puppet
A few weeks back, when I was planning and checking our route to Baler in Waze, I saw these high density of URs (update requests) in one area which sparked my interest. After driving to Baler, I know what it is now – It’s the Canili and Diayo Dam. Fortunately, SMART has data on this location, so I fired up URs to mark the start and end points of the dam so I can add it.
Please note that this stretch is one-way, depending on which side has incoming traffic first.
We stayed in one of the places in Baler along Sabang beach. So yes, route suggestions via Waze to Baler is working flawlessly now :)
We have this big website that’s currently being overhauled (means: new architecture, new tech stack and totally new code from the ground up). The lead dev asked our team if we can redirect traffic to a static site in case the actual site is down.
[Update: Our site is now launched! It’s still in beta. Check it out here: https://new.smartnet.ph]
I only overheard this but I jumped in to help because I’ve been wanting to try this feature of Route 53 but didn’t have the chance to really implement it.
I figured that that there should be a lot of tutorials on how to do this already… so this should be a walk in the park.
A little help from Google lead me to a few sites. This one is a good tutorial if you only want to redirect to different IP (steps are listed and screenshots!).
I didn’t find a good tutorial as far as aliases are involved. And we’re stuck with this loading screen:
Not really a walk in the park…
With that good tutorial as reference, we (with help from John) decided to have a crack at this ourselves.
Note: This guide assumes that your domain is already hosted in Route 53, if not you must move it first.
This how we did it:
- create a static site hosted in S3 [how?] – skip Step 3
- create your route 53 health checks [how?] – replace Step 8 with the steps below
Create a secondary alias failover using AWS CLI:
- get the Hosted zone ID of your S3 endpoint [here] – In our case we’re using Singapore so hosted id is
- get the Hosted zone ID of your domain [how?] – in this guide, let’s assume that
mysite.ph has a zone id of
- create a json file like below:
serenity:~ deadlockprocess$ cat ~/tmp/mysite.ph.json
"Comment": "mysite.ph failover",
- add the failover alias as a new record set in Route 53 with this command:
serenity:~ deadlockprocess$ aws route53 change-resource-record-sets --hosted-zone-id ABCDE12345 --change-batch file:///Users/deadlockprocess/tmp/mysite.ph.json
- you can now go back to this guide and do Step 9 onwards
- also, allow the Route 53 Health Checkers’ IPs in your firewall/security group
These guys are just wandering around Zoobic when we went there last year :)
[puppet forge] proletaryo-supervisor v0.4.0 now supports Ubuntu
Necessity is the great motivator.
I wrote this puppet module almost a year ago. The first version up to the last one only supports RedHat-based distros. Amazon Web Services is the primary platform that I use so the module is heavily tested and used in AWS Linux environments.
I was planning to support Ubuntu since day one but I managed to procrastinate because it’s not really needed in our deployments. That changed today though because we’re rolling out a few Ubuntu instances in AWS :)
I hope some people will find this module useful. It’s always open for contributions. Just fork it in GitHub:
If you have an existing Puppet installation, just install it in your Puppet Server:
puppet module install proletaryo-supervisor –version 0.4.0