When using Apache in front of Domino you need the NE rewrite attribute

Over the last week or so I have been moving to using an Apache server in front of my Domino servers. This has been working really well until I had a problem with Mark Leusink’s Auto Login’s module

The code includes some code that calls a url constructed as
/autoLogins_v113.nsf/rememberMe.xsp?to=/dev/preprodprojectlibrary.nsf

The Apache rewrite engine returns this as
/autoLogins_v113.nsf/rememberMe.xsp?to=%252Fdev%252Fpreprodprojectlibrary.nsf

whereas Domino would normally return
/autoLogins_v113.nsf/rememberMe.xsp?to=%2Fdev%2Fpreprodprojectlibrary.nsf

The solution ( via Tytus Kurek )  is to use the NE attribute in the rewrite rules as shown below.
This attribute preserves special characters such as ? and &

2014-10-23_21-29-59

Interestingly in examining what was going on it does seem as though url is being called using HTTP rather than the HTTPS protocol – I need to look into that further. This does mean that the attribute needs to be added to both the httpd.conf and the /etc/httpd/conf.d/ssl.conf files. I have updated my posts from the weekend.

Hopefully this change will not affect any other functionality – I will post if it does.

Solving the SSL V3 / SHA-2 / TLS / MD5 / Poodle problem on Linux – Part 3 – Converting a Domino SSL certificate / Private Key to Apache

Part 1 – What is the problem
Part 2 – Installing a simple Linux Reverse Proxy on the same box ( 20 mins )
Part 3 – ( This Post ) Converting your Domino SSL Certificate to work with Apache (30 mins )

Thankfully this post is now obsolete – see this IBM article which includes a modern tool to manage the Keyrings

Note that the easiest option to install a verified SSL certificate into the Apache server is to buy a new one. However, if you have a copy of Windows XP handy you can convert your existing Domino SSL certificates to Apache certificates with the following scheme.

Many thanks to Darren Duke who helped get me started with this scheme via a post on his blog about a different but overlapping scenario.

Step 0 – Why do I need to do this

Essentially you need to get your private key because Apache needs it to verify the SSL certificate,

When you originally requested your verified SSL certificate from your SSL vendor you gave them a CSR which was created via the Domino Server Certificate Administration database.

This generated a private key and used that to create the CSR request.

The SSL provider took this and using their own encryption key created the SSL certificate. This certificate will only work in the presence of your original private key and Apache cannot use the Domino .kyr file format where it is buried.

Step 1 – Install IBM ikeyman version gsk5

You need this specific version. You can download it here
It will only run on Windows XP
a) extract the zip contents to the root of the c: drive

b) Run the batch command gskregmod.bat from the command line to set some registry settings

c) Run the batch command runikeyman to open the application.

Step 2 – Extract the Private Key and Cert information from Domino the keyring KYR file

a) open the .kyr file and export it as a P12 file

2014-10-22_20-18-28

Step 3 – Convert the PK12 Format file to a text file

There are two methods here.

The simple one is to use this resource but I am never happy uploading private keys to the web.

The better method is to use the OpenSSL module already stored on your server.

Use WinSCP to copy the .PK12 file to the server and then use the following command in the Linux console

>openssl pkcs12 -in example.p12 -out example.pem

You will need to give the .PK12 password and supply a new one.

2014-10-22_20-20-12

You can now open the pem file as a text file and can see that it has 4 components ( this may vary )

1) Server Certificate Intermediate Certificate

this is the public key for the intermediate certificate and could have been obtained from the SSL certificate issuer’s web site

2) Certification Authority Root Certificate

this is the root public key for the intermediate certificate and could have been obtained from the SSL certificate issuer’s web site

3) KeyPair

This is your SSL certificate and was probably emailed to you when you purchased your SSL certificate ( but it is no good without the private key buried in the Domino kyr file )

4) Encrypted private key

This is what you are after.  I believe that you can use it in its encrypted form but you will need to provide the password every time you start Apache. To avoid this you can store it on the server in its unencrypted form but you need to make sure it is well secured otherwise a anyone who has access to it can compromise your SSL traffic.

Read section 6.6. of this document and apply the security settings recommended

You can unencrypt the key using the following command

>openssl rsa -in example.pem -out unencrypted_example.pem

2014-10-22_20-20-12

Step 5 – Deploying the Private Key and certificates

There are lots of article about how to deploy the files to Apache but this is the scheme that I used. There is another method that chains the root and intermediate into a single chain file.

a) save the intermediate certificate to the /etc/pki/tls/certs folder as a text file called intermediate.pem

You can extract the text from the file generated above or use the one sent to you when you purchased the SSL certificate.

b) save the root certificate to /etc/pki/tls/certs as a text file called root.pem

c) save your domain SSL certificate to /etc/pki/tls/certs as site_cert.crt or similar

d) save your unencrypted private key to /etc/pki/tls/private as site_private_key.pem or similar

e) modify the /etc/httpd/conf.d/ssl.conf file as follows ( allowing for any difference in file names )

SSLCertificateFile /etc/pki/tls/certs/site_cert.crt
SSLCertificateKeyFile /etc/pki/tls/private/site_private_key.pem
SSLCertificateChainFile /etc/pki/tls/certs/intermediate.pem
SSLCACertificateFile /etc/pki/tls/certs/root.pem

If you now issue the command >service hhtpd restart then the new certificates should be used.

If you are using the encrypted private key then you will be challenged for a password as shown below

2014-10-22_20-55-04

That’s it. You have now converted your Domino SSL certificate for use with Apache. Collect $200 and pass Go.

Solving the SSL V3 / SHA-2 / TLS / MD5 / Poodle problem on Linux – Part 2 – Deploying Apache with Domino

Part 1 – What is the problem
Part 2 – ( This Post ) Installing a simple Linux Reverse Proxy on the same box ( 20 mins )
Part 3 – Converting your Domino SSL Certificate to work with Apache (30 mins )

The Fix

This fix is to add an Apache Server in front of Domino where Domino is running on a Linux box. All of the web serving functionality will be handled by Apache.

It should take about 20 minutes.

Note at the end of this step you will be using a self signed SSL certificate. Part 3 shows you how to convert your Domino SSL certificate for use with Apache but note that you will need a Windows XP machine ( no kidding ).

How it works

The domino web server will be changed to server only on port 8080 ( or similar ). Apache will listen for traffic on ports 80 and 443. When requests are received they are passed through to Domino in a transparent fashion.

The port 80 traffic is configured in /etc/httpf/conf/httpd.cong
The port 443 traffic is configured in /etc/httpd/conf.d/ssl.conf file

In the example below all traffic is redirected to SSL but you can change this if you want to use both port 80 and port 443

Step 1 – Change some Domino Settings

a) In the server document change the HTTP Port to 8080 or similar.

sererdocAv2

b) In the Notes.ini settings section of the configuration document make the following changes. Adding them here means that they are less likely to get forgotten at the next upgrade etc..

HTTPAllowDecodedUrlPercent = 1
HTTPEnableConnectorHeaders = 1     IMPORTANT – please see http://nevermind.dk/nevermind/blog.nsf/subject/security-hole-leaves-ibm-domino-server-wide-open—part-two 

If you need to use the above notes ini variable then you must also add

RequestHeader unset “$WSRU”

to the httpd.conf and ssl.conf files in Apache

 

 

serverdocb

c) Check if any internet documents are set to force HTTP traffic to SSL and add a note to the Key file name field to remind you what you have done when you later have a senior moment.

serverdocC

 

d) restart the HTTP task

in the Domino console issue > tell http restart

this will free up port 80 for Apache to use.

e) Check that Domino is running on port 8080

note there may be firewall rules that need to be adjusted

http://yourserver.com:8080

Step 2 – Install Apache

Using the server console ( I use putty as it makes it easy to paste )

a) Update your Linux

>yum update

b) install Apache

>yum install httpd

c) install the SSL functionality

>yum install mod_ssl openssl

d) make Apache run on startup

>chkconfig –levels 235 httpd on

e) start Apache

>service httpd start

f) test Apache

http://yourserver.com should show the Apache page below

https://yourserver.com should throw an SSL certificate error but then open

apache

apachessleerror

 

Step 3 – Configure Apache as a reverse proxy for Domino

Apache has a lot of functionality for Aliases so that you can manage multiple web domains on a single server. I have ignored this functionality for the most part to go for a very simple but effective scheme

The easiest way to copy and edit files on Linux is to use WinSCP

a) backup the default /etc/httpf/conf/httpd.cong file

apacheA

b) in the /etc/httpf/conf/httpd.conf file add the following 12 lines

2014-10-30_11-31-54

################
ProxyRequests Off
ProxyPreserveHost On
AllowEncodedSlashes On
# these 2 lines preserve the IP address in domlog.nsf
# see http://www.leyon.at/blog/dx/getting-the-client-ip-in-an-ibm-domino-app-behind-a-reverse-proxy
SetEnvIf REMOTE_ADDR (.*) temp_remote_addr=$1
RequestHeader set “$WSRA” “%{temp_remote_addr}e”
# Include the 3 lines below if you want to always redirect to SSL
RewriteEngine on
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [NE,NC,R=301,L]
# Include the 2 lines below if you have disabled the 2 lines above
#RewriteEngine On
#RewriteRule ^/(.*) http://localhost:8080/$1 [NE,P]
################

The AllowEncoded slashes is needed for some XPages and QuickR functionality.

The RewriteCond %{SERVER_PORT} !^443$ and RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [NC,R=301,L] statement will redirect all non HTTPS traffic to HTTPS. If you do not want this to happen then comment out these lines and uncomment the next 2 lines.

The RewriteRule ^/(.*) http://localhost:8080/$1 [P] line will redirect all traffic to Domino on Port 8080. It is not required here if all traffic will be SSL as it is specified in the /etc/httpd/conf.d/ssl.conf file

Update : The NE attribute is explained in this post

c) backup the default /etc/httpd/conf.d/ssl.conf file

d) add the following 5 lines to the /etc/httpd/conf.d/ssl.conf file

2014-10-29_15-48-08

###################################
AllowEncodedSlashes On
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www.(.*)$ [NC]
RewriteRule ^(.*)$ https://%1/$1 [R=301,L]
RewriteEngine On
RewriteRule ^/(.*) http://localhost:8080/$1 [NE,P]
######################################

I was surprised that the AllowEncodedSlashes On needed to be stated again but it does as I found out when our Font Awesome functionality stopped working in XPages.

e) in /etc/httpd/conf.d/ssl.conf file disable the use of SSL V3

apached

f) restart Apache

>service httpd restart

g) test the server

http://myserver.com should redirect to https://myserver.com

the Domino content should be served up

h) test that the connection is TLS1.2 rather than SSL V3

apachee

Step 4 – Block users from accessing Domino via port 8080

blockusers

 

Conclusion

Your Domino http content is now being server via a modern web server that can use SHA-2 and TLS. It is still using the self signed SSL certificate but you can create a new one – all of the SSL providers have instructions for Apache or you can use Part 3 of the series to convert your existing SSL certificate to Apache.

Solving the Domino SSL V3 / SHA-2 / TLS / MD5 / Poodle problem on Linux – Part 1

Part 1 – ( This post ) What is the problem
Part 2 – Installing a simple Linux Reverse Proxy on the same box ( 20 mins )
Part 3 – Converting your Domino SSL Certificate to work with Apache (30 mins )

Update : IBM has published a plan to release fix packs to address these issues. Our plan is to stick with the Apache scheme ( or maybe Nginx in the future ) as it is easy to set up and gives us more flexibility.

What is the problem ?

There are a number of separate problems that can be solved ( from a Http perspective ) using the method set out here if you are running Domino on Linux

SSL3 is currently vulnerable and Domino cannot use anything newer

The Poodle exploit as detailed by Bill Malchisky means that server need to stop using SSL V3 until a fix is found. Unfortunately The newest protocol that Domino supports is SSL V3.

Certificates signed with the SLA1 algorithm are being deprecated

Google will be marking web sites protected by SHA-1 SSL certificates as suspect. This is not happening for some time but some suppliers have already stopped offering SHA-1 based SSL certificates. I suspect a lot more security audits will also mandate SHA-2 certificates.

Some SSL certificate providers will not accept Certificate Requests ( CSRs ) signed with MD5

All Domino CSRs are signed with MD5. Some SSL suppliers will not accept these Certificate Requests.

What is the solution suggested in this series of posts ?

The suggestion is to put an Apache Web server in front of Domino and have it manage all the SSL functionality. This is very simple to do.

This means that you can use SHA-2 based certificates and TLS1.2 connections rather than SSL V3. It also means that you do not need to use MD5 based CSRs

It does not help with SSL with respect to mail but there are similar mail proxys.

In this series of articles I am specifically talking about the scenario of adding an Apache server to the same Linux box as the Domino server. If you want a more sophisticated reverse proxy server managing traffic from several Domino servers then see this article from Darren Duke.

If you are even braver you could try Nginx rather than Apache

Part 2 – Installing a simple Linux Reverse Proxy on the same box ( 20 mins )

Don't try to extract the private key from the Domino.kyr file

Update – the method does work – see this post. Not sure why the technote says it doesn’t

I have been making some good progress with addressing the SSL issues by using an Apache Server in front of Domino and running on the same box – from a HTTP / HTTPS point f view it is actually pretty straight forward. One rub though is that I wanted to re-use my existing SSL certs

I think I could have done this using Windows XP ( no joke ) and IKeyman to extract the private key from the Keyring as per this technical article from IBM

How to export the private key from a Domino keyfile by using IKEYMAN

Unfortunately this method does not work and has not worked since 6.5 by the looks of it.

Under today’s theme of irony the technote contains a link to the SPR saying that the method in the technote does not work

You really couldn’t make this stuff up