Category Archives: Messaging

SSL Certificates in Zimbra 5.0.x

Using these links as a starting place,
http://www.jransomed.com/mywiki/Zimbra/InstallingSSLCertificate
http://wiki.zimbra.com/index.php?title=5.x_Commercial_Certificates_Guide

Here’s a run-down of installing SSL certificates in Zimbra Collaboration Suite (ZCS) 5.0.x. It’s relatively straightforward once you wrap your head around the steps. You’ll want to do these steps as root.

The customer on which this example is based had existing Comodo wildcard certificates for their *.domain.edu.

Browse to http://instantssl.com
log in
Arrow over to Comodo PreimumSSL Wildcard Certificate for *.domain.edu
click on ‘Download as .zip’

You should have three files that start with ‘STAR.’ rename them:
STAR_domain_edu.crt to commercial.crt
STAR_domain_edu.crt to commercial_ca.crt
STAR_domain_edu.key to commercial.key

Copy the commercial* files to each of the Zimbra hosts.


# mv commercial.key /opt/zimbra/ssl/zimbra/commercial
# mv commercial.crt commercial_ca.crt /var/tmp

Deploy the cert:


# cd /var/tmp
# zmcertmgr deploycrt comm ./commercial.crt commercial_ca.crt

This may not apply to you but we were unable to get openssl and by extension Zimbra to verify the Comodo cert chain. If zmcertmgr deploycrt is failing for you and you’re relatively confident your certs are okay here’s how I fixed it. It’s unconventional but it works. I am open to correction if someone has a more conventional fix for this..

Cd to /opt/zimbra/bin, copy zmshutil and zmcertmgr to /var/tmp and edit zmcertmgr in /var/tmp. Comment out the lines as below and add two lines also as below:


# cd /opt/zimbra/bin
# cp zmshutil /var/tmp/zmshutil
# cp zmcertmgr /var/tmp/zmcertmgr
# vi /var/tmp/zmcertmgr
    #  result=`${openssl} verify -purpose sslserver -CAfile $ca_crt $crt`
    #  if [ x"${result}" = x"${crt}: OK" ]; then
    #   echo "Valid Certificate: $result"
    echo "(artificially) Valid Certificate: $result"
    #  else
    #    echo "${ERROR_PREFIX} Invalid Certificate: $result"
    #    exit 1
    #  fi

    #  result=`${openssl} verify -purpose sslserver -CAfile $cafile $crt`

    #  if [ x"${result}" = x"${crt}: OK" ]; then
    #      echo "Valid Certificate Chain: $result"
    echo "(artificially) Valid Certificate Chain: $result"
    #  else
    #    echo "${ERROR_PREFIX} Invalid Certificate Chain: $result"
    #    exit 1
    #  fi

Once you’ve saved the modified version of zmcertmgr, run it from /var/tmp to deploy the certificates:


# cd /var/tmp
# ./zmcertmgr deploycrt comm ./commercial.crt commercial_ca.crt 
** Verifying ./commercial.crt against /opt/zimbra/ssl/zimbra/commercial/commercial.key
Certificate (./commercial.crt) and private key (/opt/zimbra/ssl/zimbra/commercial/commercial.key) match.
(artificially) Valid Certificate: 
** Copying ./commercial.crt to /opt/zimbra/ssl/zimbra/commercial/commercial.crt
** Appending ca chain commercial_ca.crt to /opt/zimbra/ssl/zimbra/commercial/commercial.crt
** Saving server config key zimbraSSLCertificate...done.
** Saving server config key zimbraSSLPrivateKey...done.
** Installing mta certificate and key...done.
** Installing slapd certificate and key...done.
** Installing proxy certificate and key...done.
** Installing CA to /opt/zimbra/conf/ca...done.

Now your certificates are installed. You need to restart Zimbra for them to take effect:


# su - zimbra -c "zmcontrol stop && zmcontrol start"

You’ll need to repeat the above for each of your servers if you have a multi-server environment: stores, mtas, ldap, etc.

If you have a relatively recent version of openssl you can test that your certificate is working by testing tls on your mta(s):


$ openssl s_client -starttls smtp -connect mta.domain.edu:25

imapsync between Zimbra environments

I spent entirely too much time in the last few days setting up to imapsync between two Zimbra environments. I will spare you the whole story but suffice to say we moved some users from production to a newly upgraded development environment and then managed to wipe out a significant portion of their mail. They were already receiving mail in the dev environment so re-copying the mail from prod wasn’t an option. Zimbra’s backup and restore isn’t smart enough to restore from one address (@prod.com->@dev.com). The obvious solution is to imapsync from prod to dev. Simple, right?

Well..

It turns out Zimbra adds headers when messages are added to the destination side. This means if you run imapsync more than once, say, to test on a folder and then run it for a whole account it will re-upload messages it just uploaded as the new headers on the destination side made the messages different. The solution is to –skipsize and ‘useheader’ enough headers so you’re sure they’ll make messages unique even if all headers aren’t in all messages (Message-ID is not in all messages for instance) but they won’t be changed on the destination side. I used –useheader Message-ID –useheader ‘From’ –useheader ‘To’ –useheader ‘Date’


imapsync --sep1 '/' --prefix1 ''  --authmech1 PLAIN --ssl1 \
    --host1 imap.prod.com --user1 user --password1 pass \
    --authmech2 PLAIN -ssl2 --host2 imap.dev.com --user2 user2 \
    --password2 pass2 --useheader Message-ID --useheader 'From' \
    --useheader 'To' --useheader 'Date' --skipsize

We also saw problems with messages presumably larger than 10mb:


++++ From [Sent] Parse 1 ++++
++++ To   [Sent] Parse 1 ++++
++++ Verifying [Sent] -> [Sent] ++++
+ NO msg #286039 [VsZ9lgHjrPxeOAn7S9sU6Q] in Sent
+ Copying msg #286039:12572025 to folder Sent
flags from : [\Seen]["04-May-2009 07:46:14 -0400"]
parse_headers want an ARRAY ref
Couldn't append msg #286039 (Subject:[0]) to folder Sent: Error trying to 
append: 6568 NO [TOOBIG] request too long

A look in the user’s sent folder for a message at 07:46 showed a 12mb message. Search the zimbra forums and:


[zimbra@store01 ~]$ zmprov gacf|grep -i zimbraMtaMaxMessageSize
zimbraMtaMaxMessageSize: 10485760
[zimbra@store01 ~]$ zmprov mcf zimbraMtaMaxMessageSize 50000000
[zimbra@store01 ~]$ 

You can now re-run imapsync as above and the message will be copied.

Also of note is –sep1 ‘/’ and –prefix ”: Apparently the Zimbra 5.0.7 imap server does not have NAMESPACE capability.

edit 090706: changed zimbraFileUploadMaxSize to zimbraMtaMaxMessageSize. According to http://wiki.zimbra.com/index.php?title=Guide_to_imapsync this was changed in 5.0.6.

Sun pam_ldap/pam_unix and correct subtree LDAP searches

Though addressed in the context JES Messaging this post is really about getting Sun’s pam_ldap or pam_unix to do what some might consider correct subtree searches. Read on:

If you’re using JES messaging with hosted domains you have an ldap tree that looks like this:
o=firstdomain.com,o=isp; o=seconddomain.com,o=isp; etc.

so your user DNs look like this: uid=morgan,ou=people,o=firstdomain,o=isp; uid=matt,ou=people,o=seconddomain,o=isp; etc.

Under normal circumstances you would supply a basedn of “o=isp,” a scope of “sub,” and the application would search all of your hosted domains in search of the username.

Sun’s pam_ldap (and I believe pam_unix) prepend “ou=people” before doing a search. So if you specify “o=isp,” pam_ldap will search within “ou=people, o=isp” which either doesn’t exist or is empty.

After searching for a way to specify multiple base dns in the ldap profile (defaultSearchBase is SINGLE-VALUE in objectclass DUAConfigProfile) it turns out there is a straightforward work-around. From the ldapclient(1) man page:


serviceSearchDescriptor
           Override the default base DN for LDAP searches  for  a
           given  service.  The  format  of  the descriptors also
           allow overriding the default search scope  and  search
           filter  for  each  service. The syntax of serviceSear-
           chDescriptor is defined in  the  profile  IETF  draft.
           The  default value for all services is NULL. This is a
           multivalued attribute. In the example,

           serviceSearchDescriptor=passwd:ou=people,dc=a1,dc=acme,dc=com?one

           the LDAP  client  would  do  a  one  level  search  in
           ou=people,dc=a1,dc=acme,dc=com       rather       than
           ou=people,defaultSearchBase for the passwd service.
 

So set serviceSearchDescriptor=passwd:o=isp and it will search under o=isp, allowing users in all of your hosted domains to authenticate. Of course this does open you up to problems where uids can conflict.

Controlling sender domain in Postfix/Zimbra 5

A client has asked that mail through his Zimbra MTA only be allowed from or to valid domains within their organization. This is particularly applicable to Zimbra as Zimbra will only archive mail if it’s from or to a domain for which it is authoritative. The idea is to archive all mail through their Zimbra environment.. If it is not one of their domains, refuse it.

If this were my organization it would look like this:
mail from user@morganjones.org to any domain would work
mail from user@1038east.com to any domain would work
mail from any domain to user@morganjones.org would work
mail from any domain to user@1038east.com would work
of course mail from and to user@morganjones.org or 1038east.com will work
all other mail will be considered relaying.

One thing we did not do that I might want to do is force authentication. The problem with this configuration is it does open up to spamming as it only validates from or to domain.

This is really a discussion about Postfix configuration but I did the work in Zimbra so I might as well add the additional steps to configure it in Zimbra.. These instructions will be applicable to straight Postfix or Zimbra.

You’ll want to do all the work as the zimbra user:
Run the zmprov command for each of your mtas.


# su - zimbra

$ zmprov ms mta01.morganjones.org zimbraMtaMyNetworks 127.0.0.0/8

$ vi /opt/zimbra/postfix/conf/main.cf
smtpd_sasl_auth_enable = no
# if you want enable sending to domains for which your environment is not
#   authoritative this is also handy for testing in your dev environment
#   that is only authoritative for a dev domain
relay_domains = 1038east.com, morganjones.org

You also want to modify smtpd_recipient_restrictions but in Zimbra you must modify that with in the zimbra configuration:


$ vi /opt/zimbra/conf/postfix_recipient_restrictions.cf
# remove permit_sasl_authenticated
check_sender_access hash:/opt/zimbra/postfix/conf/access

$ vi /opt/zimbra/postfix/conf/access
1038eaast.com OK
morganjones.org OK

$ zmmtactl reload

You might want to check that /opt/zimbra/postfix/conf/main.cf now contains this:


smtpd_recipient_restrictions = reject_non_fqdn_recipient,
check_sender_access
hash:/opt/zimbra/postfix/conf/access, permit_mynetworks,
reject_unauth_destination, reject_unlisted_recipient,
reject_invalid_hostname, reject_non_fqdn_sender, permit

You should now be set.

It’s worth mentioning: check_sender_access will only check and allow the sender domain. if you don’t set relay_domains the recipient domain is allowed because your environment is the final destination for that/those domain(s). As noted above you can set relay_domains above if you want to allow relaying to domains for which this environment is not the final destination.

Zimbra LDAP Debugging

Multi-node Zimbra installs sometimes fail in mysterious ways.. We recently resolved what turned out to be a network problem but it was causing our Zimbra install to fail with what I originally suspected was an LDAP problem. I think the troubleshooting process may prove useful. This is Zimbra 5.0.4:

If a store doesn’t appear to be communicating with its ldap master, here’s how a I debugged it

On the ldap master:


# vi /etc/syslog.conf
    local4.debug          -/var/log/zimbra.log
# /sbin/service syslog reload
Reloading syslogd...                                       [  OK  ]
Reloading klogd...                                         [  OK  ]
# su - zimbra
$ zmlocalconfig -e ldap_log_level=800
$ zmcontrol stop && zmcontrol start

Now tail -f /var/log/zimbra.log for slapd logging

Now from the store:

yum install openldap-clients (RHEL5) or
up2date openldap-clients (RHEL4) if ldapsearch isn’t installed


$ ldapsearch -h zldap.morganjones.internal -W -x -LL -D cn=config
-b cn=zimbra objectclass=*
Enter LDAP Password:
version: 1 

dn: cn=zimbra
objectClass: organizationalRole
description: Zimbra Systems Application Data
cn: zimbra 

dn: cn=admins,cn=zimbra
objectClass: organizationalRole
description: admin accounts
cn: admins 

...

dn: cn=com_zimbra_convertd,cn=zimlets,cn=zimbra
zimbraZimletDescription: Convertd Extension for Admin UI
zimbraZimletVersion: 1.0
objectClass: zimbraZimletEntry
zimbraZimletIndexingEnabled: TRUE
zimbraZimletKeyword: com_zimbra_convertd
cn: com_zimbra_convertd
zimbraZimletIsExtension: TRUE
zimbraZimletPriority: 12
zimbraZimletEnabled: TRUE
$

side note: Zimbra users TLS for connections before stores and ldap servers. ‘-LL’ forces ldapsearch to use TLS, -x turns off ldaps.

Here’s the background that started me down this path:

Install ldap master with at least zimbra-ldap

Install a store, answer ‘n’ to zimbra-ldap and ‘y’ to zimbra-store. At the Main menu choose ‘1’ for Common Configuration.

Set Ldap master host and Ldap Admin password and when I typed ‘r’ it hung just like this:


Common configuration

   1) Hostname:                                store01.morganjones.internal
   2) Ldap master host:                      zldap.morganjones.internal
   3) Ldap port:                                389
   4) Ldap Admin password:                 set
   5) LDAP Base DN:                           cn=zimbra
   6) Require secure interprocess communications: yes
   7) TimeZone:
             (GMT-05.00) Easten Time (US & Canada)

Select, or 'r' for previous menu [r] r

A quick look at /tmp/zmsetup* revealed:


Couldn't bind to zldap.morganjones.internal as uid=zimbra,cn=admins,cn=zimbra
Checking ldap on zldap.morganjones.internal:389
Unable to startTLS: Resource temporarily unavailable
Couldn't bind to zldap.morganjones.internal as uid=zimbra,cn=admins,cn=zimbra
checking isEnabled zimbra-store

Aha.. an LDAP connectivity problem.

Canonical address configuration in multi-server Zimbra

The Zimbra environment we’ve put together looks like this:
1 LDAP master
3 MTAs
6 stores

We initially installed with only mta instances on the mtas and all hosts pointing to the LDAP master.
This was of course lacking ldap consumers on the MTAs.

So a few weeks ago we upgraded to Zimbra 4.5.6 and added LDAP consumers to the MTAs. This was messy but reliable once you take your lumps and jump through the hoops in the right order. I owe a post on this topic.

Our Zimbra environment is not in production yet, it’s officially pre-production, so we use canonical addresses to make our pre-production users’ mail look to come from productiondomain.com instead of pre-productiondomain.com.

Immediately after adding LDAP consumers to the MTAs our pre-prod users began to complain that their mail was coming from pre-productiondomain.com. This was prematurely releasing our fancy new pre-productiondomain.com which is supposed to be a surprise for some time in the fall.

So.. Zimbra uses canonical maps for the canonical address setting:

$ grep canonical /opt/zimbra/postfix/conf/main.cf
sender_canonical_maps = ldap:/opt/zimbra/conf/ldap-scm.cf
$ cat /opt/zimbra/conf/ldap-scm.cf
server_host = ldap://master_ldap.pre-productiondomain.com:389
server_port = 389
search_base = 
query_filter = (&(|(zimbraMailDeliveryAddress=%s)(zimbraMailAlias=%s)
(zimbraMailCatchAllAddress=%s))(zimbraMailStatus=enabled))
result_attribute = zimbraMailCanonicalAddress,
zimbraMailCatchAllCanonicalAddress
version = 3
bind = no
timeout = 30
$

You should immediately notice that postfix is looking to the master for Canonical address rewriting. This may be a side effect of adding an LDAP consumer after the MTA was installed. Either way it’s wrong. It easily tested:


$ ldapsearch -x -h master_ldap.pre-productiondomain.com -b '' 
'(&(|(zimbraMailDeliveryAddress=morgan@pre-productiondomain.com)
(zimbraMailAlias=morgan@pre-productiondomain.com)
(zimbraMailCatchAllAddress=morgan@pre-productiondomain.com))
(zimbraMailStatus=enabled))' zimbraMailCanonicalAddress 
zimbraMailCatchAllCanonicalAddress

You should get nothing. Repeat the command with -h mta_hostname.pre-productiondomain.com and you should get the canonical address you set in the admin gui.

The solution is to edit /opt/zimbra/conf/ldap-scm.cf and change
server_host to ldap://mta_hostname.pre-productiondomain.com:389

If you tried it and it still doesn’t seem to be working, you might have zimbraPrefFromAddress set. On one of the stores:


$  zmprov ga morgan@pre-proddomain.com|grep -i zimbraPrefFromAddress
zimbraPrefFromAddress: morgan@pre-proddomain.com
$

Fix it by unsetting the attribute:


zmprov ma morgan@pre-proddomain.com zimbraPrefFromAddress ''

It seems to take a few minutes to take effect.

Hidden Download: JES4 or Messaging 2005q4

Or where do I download java_es_05Q4-ga1-solaris-x86-1-iso.zip, java_es_05Q4-ga1-solaris-x86-2-iso.zip, java_es_05Q4-ga1-solaris-sparc-1-iso.zip and java_es_05Q4-ga1-solaris-sparc-2-iso.zip?

Sun has released Java Enterprise System 5 (JES5), comprised of Messaging 6.3 and Directory 6.0. The upgrade from JES4 or Messaging and Directory 2005q4 is pretty well documented.

What Sun does not document is where an integrator like myself would download a copy of 2005q4 to test upgrades in preparation for upgrading for a customer.

Their all downloads page lists iPlanet 5.2 (the version before 2005q4) and Java Enterprise System 5 or JES5 (the version after 2005q4).

It turns out it’s available on bigadmin.

Automating Zimbra Installs

..or Zimbra silent install in Sun terminology.

This was tested on Zimbra 4.5.4 Network edition, RHEL4 but is probably applicable to all 4.5.x versions.

I’m talking about installing Zimbra components automatically rather than running the install script and answering a bunch of questions. A silent install script can be put into kickstart scripts to automate the Zimbra install during server provisioning. This is handy if you have a server failure or you need to provision additional machines to scale your infrastructure. It takes out the guesswork and speeds up the process.

The Zimbra architecture I’m working on currently is a multi-server install without clustering. We have 1 ldap master, 4 mtas, 5 stores and 2 multiplexers. Each uses the Zimbra installer but the questions are answered differently depending on the type of host.

You can automate the install by:
– using a generated config.pid file or
– capturing your keystokes and playing them back.

If you are doing a multi-server install with replicated ldap, you will not be able to use the generated configure to install your stores. This is because you have to disable ldap during the store install or the install will fail. The configure doesn’t record that you disabled ldap. So:


# gzip -dc zcs*NETWORK*z|tar xf -
# cd zcs
# ./install.sh

Open an editor in a second window and each time you type a key during your install type that key into your editor.

Here’s one of my keystrokes_host_install.in files (with domain name changed to my own to protect the innocent):
This example starts with a carriage return



y
y
n
y
y
y
y
n
2
ldap.1038east.com
4
pass
6
4
pass
9
smtp.1038east.com
20
/home/morgan/ZCSLicense.xml
r
5
1
r
a
y

y

Now uninstall (./install -u) and:


./install.sh < ../keystrokes_host.in

References
forum posting on automated installation.posting
forum posting on ldap replication

lighttpd and mailman

There does not appear to be a how-to about integrating mailman with lighttpd. I’m relatively new to lighttpd and it’s pretty different from Apache. Please let me know if I am missing something significant.

I am using mailman 2.1.9 installed from rpm in Fedora Core 6 (fc6). There should not be much difference if you install it from the distribution.

If you install mailman from scratch it is important that you run the configure with –with-cgi-gid=lighttpd. Substitute ‘lighttpd’ with the group id you will be using to run lighttpd. lighttpd runs as group ‘lighttpd’ by default in fc6.


# mkdir /srv/www/lighttpd/images
# cp /usr/lib/mailman/icons/* /srv/www/lighttpd/images
# vi /usr/lib/mailman/Mailman/mm_cfg.py
    IMAGE_LOGOS = '/images/'
# vi /etc/lighttpd/lighttpd.conf


uncomment the following from "server.modules:"


    "mod_redirect",
    "mod_cgi",


If you're running lighttpd and mailman from the Fedora RPM:


    server.groupname            = "apache"


then


    # Exec        /mailman/*      $prefix/cgi-bin/* or
    # ScriptAlias /mailman/       $prefix/cgi-bin/
    alias.url = ( "/mailman" => "/usr/lib/mailman/cgi-bin",
        "/pipermail/" => "/var/lib/mailman/archives/public")
    $HTTP["url"] =~ "^/mailman" {
        cgi.assign = ( "" => "" )
    }

The Apache config directives are in comments above. In essence:
Alias /mailman to "/usr/lib/mailman/cgi-bin" on the filesystem
Tell lighttpd that any web path starting with /mailman contains executables.
cgi.assign = ("" => "") tells lighttpd that files without extensions should be run without an interpreter.

Zimbra install fails due to missing /usr/lib64/libstdc_++.so.5

We are running Zimbra 4.5.3 Network Edition on Sun x4200s and x4100s with RHEL4. We’ve fully patched Redhat and I’m 99% sure certain Redhat was installed with full 64 bit support.


# gzip -dc zcs-NETWORK-4.5.3_GA_733.RHEL4_64.tgz |tar xf -
# cd zcs
# ./install.sh
...
Checking for prerequisites...
    NPTL...FOUND
    sudo...FOUND sudo-1.6.7p5-30.1.3
    libidn...FOUND libidn-0.5.6-1
libidn-0.5.6-1
    curl...FOUND curl-7.12.1-8
curl-7.12.1-8
    fetchmail...FOUND fetchmail-6.2.5-6
    gmp...FOUND gmp-4.1.4-3
gmp-4.1.4-3
    compat-libstdc++-296...FOUND compat-libstdc++-296-2.96-132.7.2
    compat-libstdc++-33...FOUND compat-libstdc++-33-3.2.3-47.3
    /usr/lib/libstdc++.so.5...FOUND
    /usr/lib64/libstdc++.so.5...MISSING

###ERROR###

One or more prerequisite packages are missing.
Please install them before running this installer.

Installation cancelled.

#

The solution is relatively simple:


# up2date --arch=x86_64 -i compat-libstdc++-33

Fetching Obsoletes list for channel: rhel-x86_64-es-4...

Fetching rpm headers...
########################################

Name                                    Version        Rel     
----------------------------------------------------------
compat-libstdc++-33    3.2.3    47.3     x86_64


Testing package set / solving RPM inter-dependencies...
########################################
compat-libstdc++-33-3.2.3-4 ########################## Done.                   
Preparing              ######################################
##### [100%]

Installing...
   1:compat-libstdc++-33    
########################################### [100%]
#

Redhat doesn't name their 64 libraries differently than their 32 bit libraries. I expect there's a way to tell the difference.