morgan::blog

June 30, 2009

imapsync between Zimbra environments

Filed under: Messaging, Zimbra — morgan @ 15:33

I spent entirely too much time in the last few days setting up to imapsync between two Zimbra environments. I will spare you the whole story but suffice to say we moved some users from production to a newly upgraded development environment and then managed to wipe out a significant portion of their mail. They were already receiving mail in the dev environment so re-copying the mail from prod wasn’t an option. Zimbra’s backup and restore isn’t smart enough to restore from one address (@prod.com->@dev.com). The obvious solution is to imapsync from prod to dev. Simple, right?

Well..

It turns out Zimbra adds headers when messages are added to the destination side. This means if you run imapsync more than once, say, to test on a folder and then run it for a whole account it will re-upload messages it just uploaded as the new headers on the destination side made the messages different. The solution is to –skipsize and ‘useheader’ enough headers so you’re sure they’ll make messages unique even if all headers aren’t in all messages (Message-ID is not in all messages for instance) but they won’t be changed on the destination side. I used –useheader Message-ID –useheader ‘From’ –useheader ‘To’ –useheader ‘Date’


imapsync --sep1 '/' --prefix1 ''  --authmech1 PLAIN --ssl1 \
    --host1 imap.prod.com --user1 user --password1 pass \
    --authmech2 PLAIN -ssl2 --host2 imap.dev.com --user2 user2 \
    --password2 pass2 --useheader Message-ID --useheader 'From' \
    --useheader 'To' --useheader 'Date' --skipsize

We also saw problems with messages presumably larger than 10mb:


++++ From [Sent] Parse 1 ++++
++++ To   [Sent] Parse 1 ++++
++++ Verifying [Sent] -> [Sent] ++++
+ NO msg #286039 [VsZ9lgHjrPxeOAn7S9sU6Q] in Sent
+ Copying msg #286039:12572025 to folder Sent
flags from : [\Seen]["04-May-2009 07:46:14 -0400"]
parse_headers want an ARRAY ref
Couldn't append msg #286039 (Subject:[0]) to folder Sent: Error trying to 
append: 6568 NO [TOOBIG] request too long

A look in the user’s sent folder for a message at 07:46 showed a 12mb message. Search the zimbra forums and:


[zimbra@store01 ~]$ zmprov gacf|grep -i zimbrafileuploadmaxsize
zimbraFileUploadMaxSize: 10485760
[zimbra@store01 ~]$ zmprov mcf zimbraFileUploadMaxSize 50000000
[zimbra@store01 ~]$ 

You can now re-run imapsync as above and the message will be copied.

Also of note is –sep1 ‘/’ and –prefix ”: Apparently the Zimbra 5.0.7 imap server does not have NAMESPACE capability.

April 24, 2009

Bear with me..

Filed under: Uncategorized — morgan @ 7:27

I’m having theme issues, a consistent look will return shortly.

March 27, 2009

Photo and audio from JJ

Filed under: Casinos — morgan @ 17:35

http://www.jjtiziou.net

March 11, 2009

Zimbra Attachment Parsing Bugs

Filed under: Uncategorized — morgan @ 20:04

After a morning of troubleshooting poor performance on an archive store that was causing backups on the mtas I would bet we’ve hit a regression. Zimbra doesn’t recognize it.

One of the admins IM’ed me a little after 9am yesterday complaining of queue backups on the mtas delivering to the archive store. They have 5 user stores and 1 archive store.

Connecting to port 7025 (lmtp) on the archive store shows sporadic hangs. Some mail was getting in but thousands were backed up on the mtas.

A quick look at the mailbox.log showed errors connecting to the conversion daemon:

<pre> 2009-03-05 10:25:54,909 WARN [LmtpServer-20] [name=user-20080704@domain.tld.archive; mid=70;] ParsedMessage - Unable to parse part 2 (file.pdf, application/pdf) of message with Message-ID <000f01c29c4e$3434d3c0$315c200a@user.domain.tld>. com.zimbra.cs.mime.MimeHandlerException: cannot extract text at com.zimbra.cs.mime.handler.DefaultHandler.getContentImpl( DefaultHandler.java:117) at com.zimbra.cs.mime.MimeHandler.getContent(MimeHandler.java:148) at com.zimbra.cs.mime.ParsedMessage.analyzePart(ParsedMessage.java:1148) at com.zimbra.cs.mime.ParsedMessage.analyzeNonBodyParts(ParsedMessage.java:426) ... Caused by: com.zimbra.cs.convert.ConversionException: Cannot extract text from document that previously crashed convertd. at com.zimbra.cs.convert.TransformationClient.extract( TransformationClient.java:196) at com.zimbra.cs.convert.SocketTransformationStub.doExtract( SocketTransformationStub.java:84) at com.zimbra.cs.convert.SocketTransformationStub.extract( SocketTransformationStub.java:116) at com.zimbra.cs.mime.handler.DefaultHandler.getContentImpl( DefaultHandler.java:111) ... 27 more </pre>

So on first look convertd died, I restarted it:
su - zimbra -c "zmconvertctl start"

convertd started and the errors in mailbox.log went away. But kvoop processes started to consume cpu. The load rose to almost 30.

This is the kvoop bug we fixed by turning of attachment indexing.. So I checked that and archiving is on again on the archive cos(!)


zimbra@host log]$ zmprov gc default |grep -i attachment
zimbraAttachmentsBlocked: FALSE
zimbraAttachmentsIndexingEnabled: FALSE
zimbraAttachmentsViewInHtmlOnly: FALSE
[zimbra@host log]$ zmprov gc archive |grep -i attachment
zimbraAttachmentsBlocked: FALSE
zimbraAttachmentsIndexingEnabled: TRUE
zimbraAttachmentsViewInHtmlOnly: FALSE
[zimbra@host log]$ 

zimbra@host log]$ zmprov modifyCos archive zimbraAttachmentsIndexingEnabled 
FALSE
zimbra@host log]$

I flushed the queues and all is well.

Background:

http://wiki.zimbra.com/index.php?title=Disable_Indexing_for_Specific_File_Types


http://bugzilla.zimbra.com/show_bug.cgi?id=14016

Look at the bottom of the bugzilla link: It looks to me like we’re hitting a regression: a bug they thought they fixed in 4.5.x is resurfacing in 5.0.x:


------- Comment  #10 From tech 2008-10-06 03:44:08  -------

this bug seems to have started happening intermittently on my 5.04 rhel_x86
installation it hangs the server so i have to reboot

RSI Log: weeks of 2/2 and 2/9

Filed under: repetitive strain injury (RSI) — morgan @ 19:51

I spent the week of 2/2 I spent on site with a customer. I took my Kinesis keyboard and iFold stand and various hand rests with me. I was in more consistent pain than I can remember for most of the week. I experimented with various heights of wrist wrests by inserting dense foam (designed for motorcycle race seats). Ultimately I think my chair was too low. Sit in your chair, put your elbows at your sides bent 90 degrees and shrug your shoulders–that was about the level of the keyboard.

2/9 I was back to my regular desk in where I am able to raise the chair high enough that the keyboard is right below my hands with my elbows at my side and bent 90 degrees. That and a gel support for my mouse hand is working well and I am maybe even typing faster than normal. I am not using wrist rests at the keyboard: in theory with my elbows at my sides I am suspending only my hands and forearms. Bellis and Damany advocate no wrist rests.

The nerve “twanginess” is all but gone, replaced by some pain in my elbows around the ulnar nerve. Bending my elbow is consistently less painful this week: I can button a shirt, put on a motorcycle helmet, braid my hair etc, with only minimal pain.

I am finding when I sleep I notice very slight numbness in my hands more than usual, about half the time in the ulnar distribution. It’s slight to the point that it might be in my head. I am sleeping with B more nights than not and she often sleeps on one of my arms.. I find I can rotate my arm slightly or shift my weight to a more bony part of my arm to make the numbness go away.

I discovered this weekend that I have traded sleeping on my arm with my elbow bent for leaning on or contorting my wrist. I’ll need to work on that.

February 6, 2009

Sun pam_ldap/pam_unix and correct subtree LDAP searches

Filed under: Directory/LDAP, JES, Java Enterprise System, Messaging, Solaris 10 — morgan @ 0:26

Though addressed in the context JES Messaging this post is really about getting Sun’s pam_ldap or pam_unix to do what some might consider correct subtree searches. Read on:

If you’re using JES messaging with hosted domains you have an ldap tree that looks like this:
o=firstdomain.com,o=isp; o=seconddomain.com,o=isp; etc.

so your user DNs look like this: uid=morgan,ou=people,o=firstdomain,o=isp; uid=matt,ou=people,o=seconddomain,o=isp; etc.

Under normal circumstances you would supply a basedn of “o=isp,” a scope of “sub,” and the application would search all of your hosted domains in search of the username.

Sun’s pam_ldap (and I believe pam_unix) prepend “ou=people” before doing a search. So if you specify “o=isp,” pam_ldap will search within “ou=people, o=isp” which either doesn’t exist or is empty.

After searching for a way to specify multiple base dns in the ldap profile (defaultSearchBase is SINGLE-VALUE in objectclass DUAConfigProfile) it turns out there is a straightforward work-around. From the ldapclient(1) man page:


serviceSearchDescriptor
           Override the default base DN for LDAP searches  for  a
           given  service.  The  format  of  the descriptors also
           allow overriding the default search scope  and  search
           filter  for  each  service. The syntax of serviceSear-
           chDescriptor is defined in  the  profile  IETF  draft.
           The  default value for all services is NULL. This is a
           multivalued attribute. In the example,

           serviceSearchDescriptor=passwd:ou=people,dc=a1,dc=acme,dc=com?one

           the LDAP  client  would  do  a  one  level  search  in
           ou=people,dc=a1,dc=acme,dc=com       rather       than
           ou=people,defaultSearchBase for the passwd service.
 

So set serviceSearchDescriptor=passwd:o=isp and it will search under o=isp, allowing users in all of your hosted domains to authenticate. Of course this does open you up to problems where uids can conflict.

February 5, 2009

RSI log

Filed under: repetitive strain injury (RSI) — morgan @ 0:14

Bellis and Damany recommend keeping a log. Here’s a quick two month summary:

Prior to 11/29/08: I remember periodic pain in the palm of my hands below the ring and pinky finger, particularly when using a laptop. My first memory of this is around October, 06. Infrequent pain in my forearms along the path of the ulnar nerve (elbow to base of palm).

11/29/08: I noticed consistent pain from my elbow to the base of my palm for several days straight over the Thanksgiving holiday. I searched the Internet and self diagnosed cubital tunnel syndrome. I immediately began sleeping with my arms straight. When I returned I went to my physician and she referred me to a hand surgeon.

About a week later: my pain changed to consistent pain in my elbow with periodic nerve “twanginess” from my elbow into my hand: imagine the feeling immediately after the initial shock of hitting your funny bone all the time in both arms. This was consistent until mid January.

Bending my arms any more than 30 degrees is a 6-7 in pain and I have to brace myself mentally to button my shirt, touch my face, drink from a glass, braid my hair, wash my hair, etc.

12/9/08: (check date) I met with a surgeon at the Hand Center at Jefferson. She confirmed my diagnosis, told me to continue sleeping with my arms straight, talked a little too much about surgery for my tastes and ordered an EMG to assess the severity of the damage to my nerves.

12/23/08: The EMG verified the diagnosis of cubital tunnel but indicated marginal severity: there was damage but it was light to moderate. It also showed mild carpal tunnel syndrome. I met again with the surgeon and she prescribed physical therapy and asked that I return in six months for another EMG. I remember telling her that I felt I had about a 10% improvement in pain but couldn’t be sure as consistent pain can be hard to gauge. I also gained some confidence in the surgeon that day as she was no longer talking about surgery: she was taking a conservative approach.

It was at about this time that I began taking 400mg of Aleve twice a day: at breakfast and dinner.

1/5/09: I met with the physical therapist, she found no measurable loss of sensation or motor function. Loss of sensation and motor function are the next step in the syndrome and I am not there… I told her I felt I had improved an additional 10% but was still not sure. She gave me median nerve glides which also work the ulnar nerve a little, hand exercises for carpal tunnel syndrome. She also asked I return the following week with photos of where I work so we could evaluate ergonomics.

1/12/08: I returned to physical therapy with photos of me on the couch, feet up, laptop in my lap, slouched in front of my desk, sitting at the kitchen table and so on. She was appalled but not surprised. She showed me the proper seating position and prescribed a split keyboard, a laptop stand and a keyboard tray. I’m tall enough that the key board tray has proven unnecessary but I am now an expert on and believer in split keyboards. I currently own two. I type with almost no hand pain now. If I remember correctly I was still in pain in my elbows more often than not but it was less at times. It is still uncomfortable to bend more than 30 degrees.

1/28/08: I returned to physical therapy with my Goldtouch keyboard and photos of me in a correct position. J, my physical therapist approved. I am now pain free 10% of the time, have nerve twanginess/awareness of the ulnar nerve from my elbow into my hand about 70% of the time and 4-5 pain 20% of the time. Activities requiring extreme arm bending (buttoning my shirt, braiding my hair, washing my hair, rubbing my face) have gone from a 6-7 in pain to a 1-2. I would say I notice these activities now rather than bracing for them.

2/4/08: I am pain free about 60-70% of the time. I seldom have serious pain or awareness of the full run of the ulnar nerve from my elbow into my hand. I have minor pain in my hand, sometimes my forearm. Activities involving extreme elbow bending still hurt about a 1-2.

reading and more insight into RSI

Filed under: repetitive strain injury (RSI) — morgan @ 0:00

I’m reading Bellis and Damany’s It’s Not Carpal Tunnel Syndrome! RSI Theory & Therapy for Computer Professionals. Their thesis is that many RSIs won’t be solved by surgery, at least not without an understanding of the larger system and it is possible to properly treat an RSI without giving up computer use.

Bellis is a computer professional who had surgery, his symptoms came back only to be resolved by working with an experience physical therapist: Damany. Their advice is in line with what my physical therapist is recommending and it’s working for me. I am now just over two months into my treatment of my RSIs and I am seeing improvement.

Of particular interest is the idea that just fixing an RSI through surgery, though often providing immediate relief will not provide long term relief if you don’t address the underlying cause of the damage. The hands, arms, back, neck, etc. work as a system: just operating on the wrist or the elbow doesn’t address the rest of the system. Further, in many cases RSI can be relieved by just addressing the cause and skipping the surgery altogether. That speaks to me.

Particularly interesting about nerve damage is there is really no way to strengthen or condition nerves. Physical therapy for muscles usually involves strengthening and the result is progressive rebuilding. Physical therapy for nerves involves moving your [arms in my case] in a way to take them through their full range of motion but otherwise just modifying behavior to not put stress on the nerves and hope they heal. And they do heal but over a course of months.

Sensory feedback from muscles is consistent: they hurt when you exercise them and then they heal.. While nerves presumably heal progressively they send confusing, misleading messages about what is going on: one day it will be pain in the hand, another it will be itching “inside” your hand that can’t be scratched. Some days I’ll work most of the day on the keyboard in pain only to have it clear up three quarters of the way through the day and not hurt at all the rest of the day even though I’m working as much as I was in the morning.

I am also a little surprised about how little modern medicine knows about RSIs.. Part of the problem according to Bellis and Damany is that modern medicine treats the body as regions and doctors are trained to focus on one part and so don’t always look at the big picture before treating the location of the pain: damage to the median nerve where it passes through the carpal tunnel for instance is often the result of bad posture and if you operate but don’t’ address the bad posture you will end up with the same symptoms sometimes in a matter of weeks.

It’s your responsibility as a patient to talk to multiple people and try to understand what is going on and help guide your treatment.. Your doctor may well know less about your condition than you do.. Bellis and Damany suggest physical therapy with a focus on the whole system and not just the wrist or just the elbow the most effective way to treat an RSI.

January 12, 2009

ergonomics

Filed under: repetitive strain injury (RSI) — morgan @ 23:20

We addressed computer ergonomics today in physical therapy. I am of course a mess. The hardest part of all of this is how tightly the convenience of a laptop is intertwined into my life.. My early feelings on the matter are that I’m going to have have to set up my desk at home and any office I work for more than a day or two. It’s vaguely depressing to drag around a split keyboard and stand everywhere I go but I keep reminding myself it beats surgery..

Here’s an example of what she recommended:
http://www.medicinenet.com/script/main/art.asp?articlekey=22781

Basically: sit back in a chair with lumbar support. It should push your shoulders back. Your knees should be 90 degrees, your ankles should be 90 degrees. Your chair should of course be high enough to allow your knees to be 90 degrees. The top of the monitor should be level with your forehead. Your arms should be at your sides, 90-100 degree bend in your elbows, wrists neutral. My PT says you usually want a keyboard tray to place the keyboard at the right height but in my case I may be able to get away without one because I’m tall enough to reach the keyboard on a table and maintain a neutral wrist position. Hands should bend in naturally.. You want a keyboard that takes minimal finger effort to type on. You should use as little force as possible while typing.

What this means for a traveling laptop user is a portable notebook stand and a split keyboard:
http://store.ergocube.com/gotrnost5.html and
http://www.amazon.com/Goldtouch-Apple-Compatible-Keyboard-Black/dp/B001IIP9UY/ref=sr_1_1?ie=UTF8&s=electronics&qid=1231819997&sr=8-1

You can buy them both at goldtouch.com but their return policy is draconian and I have never seen either in person.

Yes, that’s a Mac keyboard.. draw your own conclusions. No, it hasn’t arrived yet.

A conventional mouse is fine.. so I’m just going to carry one of of the dozen or so Dell mice hanging in my closet.. eventually I’ll pick up a bluetooth mouse.

Traditional molded ergonomic keyboards are apparently bad: they’re one size fits all, ie don’t allow adjustment. Keyboards that tilt toward you are bad. If a keyboard has feet you’re better off folding them down and laying it flat.

I find it surprising that no one makes a bluetooth split keyboard. Goldtouch will have a travel split keyboard in February: http://www.goldtouch.com/p-139-goldtouch-go-travel-keyboard.aspx

January 6, 2009

Join Casino Free Philadelphia for the launch of their latest campaign tomorrow

Filed under: Casinos — morgan @ 14:24

Please join us tomorrow, Wednesday evening, from 6-8:30pm at the offices of Liberty Resources, 714 Market Street, as we unveil our newest campaign during this critical phase.

As you know, we have been working for some time to brainstorm and plan. We’ve talked to many of you; we’ve analyzed the status of the various permit applications, the financial situation and the legal landscape. We’ve listened and we’ve even adjusted our game plan. Based on the external events, we believe our movement now has a short but incredibly valuable window of opportunity — about 6 months — to change hearts and minds. And, that’s just what we’re going to do, with your help and dedication.

Join us tomorrow, Wednesday evening, from 6-8:30pm, as we Declare Our Independence from Casinos. Join us to learn about our upcoming search for an elected official who will stand with us. Join us as we prepare to draft our manifesto against casinos in our City. Join us as we set out to organize citizens from all across Philadelphia.

At the meeting we’ll present to you an ambitious, necessary and forceful campaign; and we’ll continue to incorporate your input. We will provide an opportunity for you to plug into our efforts and become responsible for specific tasks. The time is right for you to become even more involved, even more committed and even more protective of your beloved City.

Bring your neighbors, bring your friends and bring your passion.

Warmly,

Amy, Ivan, Jethro and the entire team at Casino-Free Philadelphia

Older Posts »

Powered by WordPress