Using Docker and ELK to Analyze WebSphere Application Server SystemOut.log

I often get SystemOut.log files from customers or friends to help them analyzing a problem. Often it is complicated to find the right server and application which generates the real error, because most WebSphere Applications (like IBM Connections or Sametime) are installed on different Application Servers and Nodes. So you need to open multiple large files in your editor, scroll each to the needed timestamps and check the lines before for possible error messages.

Klaus Bild showed on several conferences and in his blog the functionality of ELK, so I thought about using ELK too. I started to build a virtual machine with ELK Stack (Elasticsearch, Logstash & Kibana) and imported my local logs and logs i got mailed. This way is cool to analyze your environment, but adding just some SystemOut.logs from outside is not the best way. It’s hard to remove this stuff after analyzing from a ELK instance.

Then I found a tutorial to install ELK with Docker, there is even a great Part 2 of this post, which helps us installing an ELK Cluster. Just follow this blog posts, install Docker and Docker-compose. It’s really fast deployed. In my case i do not use flocker, it’s enough to use a local data container for the elasticsearch data.

Why do I use Docker instead of a virtual machine? It’s super easy to just drop the data container and begin with an empty database.

My setup

I created a folder on my Mac to put all needed stuff for the Docker images to it:

mkdir elk
cd elk

(more…)

IBM Connections 5.5 install Ephox Editors

Several people told me that installing the Editors is not described very well in the IBM Connections documentation. So i decided to write down the steps I used to deploy the editors. Hope it helps.

Check Installation on ephox: http://docs.ephox.com/display/EphoxForIBMConnections/Installing+Textbox.io+Services

  • Extract EPHOX_EDITORS3.0.1_CONN5.5.zip
    • edit config/config.js

      • editor: 'textbox.io | EditLive | CKEditor | role-based'
        This sets the default editor. When you use role-based you can enable textbox.io and EditLive for different user groups.
        Be aware that editlive needs a Java plugin which is mostly deactivated or outdated in actual browsers.
      • Enable Spelling-Servie URL (you need to deploy tbioServices_c5.ear):
        spellingServiceUrl: "https://connections-host/ephox-spelling"
        You have to set to https, that spell-checking works with http and https access to Connections.
        If you have selected role-based, you must deploy the EphoxEditorsForConnections.ear
      • When you want to use the builtin spellchecking, you need to install services/tbioServices_c5.ear
    • Start ./install.sh or install.exe

      • ./install.sh root@webspherehost <customization-dir> <webressources-dir>
        You need to type the root password 3 times, because installation uses seperate ssh calls for the installation
  • Create /opt/ephox/application.conf (WINDOWS: WAS_INSTALLATION_DRIVE:\opt\ephox\application.conf)
        
ephox {
     allowed-origins {
       origins = [
          "https://connections-55.panastoeps.local",
          "http://connections-55.panastoeps.local"
       ],
       url = "https://connections-55.panastoeps.local/ephox-allowed-origins/cors"
     }
}
  • Map Applications to your webserver, update the Connections versionstamp and restart Common and Ephox Applications
  • Verify: http://<your server and port>/connections/resources/web/ephox.editors.connections/verify.html

Missing images in Wikis after migration to IBM Connections 5.5

Wikis in IBM Connections 5.5 have a little bug, because the link (/library instead of /wikis/form/api/library) for images are wrong and so they are not displayed.

There is a technote, which should solve this issue, but the used way with ProxyPass is not what i want to use. When you use ProxyPass and ProxyPassReverse you should add a ProxyRequest off to be more secure. ProxyPass to localhost can be a problem too, i would suggest to change localhost to the Connections IHS Hostname.

Why do I use a different approach?

Most of my deployments already use a RewriteRule to redirect the hostname to there Connections Homepage, so i don’t need an additional module (which needs ressources and can have security considerations), when i can solve the image issue through mod_rewrite.

RewriteRule "^/library/(.*)" "/wikis/form/api/library/$1" [R,L]

If you haven’t set <forceConfidentialCommunications enabled="true"/> in LotusConnections-config.xml you need to set the RewriteRule and the ProxyPass config within your http and your https configuration parts!

Example httpd.conf:

... 
# HTTP configuration
<VirtualHost *:80>
    ServerName connections.example.com

    RewriteEngine On

     # Redirect hostname to Homepage
    RewriteRule ^\/$ https://connections.example.com/homepage [noescape,L,R]

    # Fix wrong wiki image URL
    RewriteRule "^/library/(.*)" "/wikis/form/api/library/$1" [R,L]
</VirtualHost>
# HTTPS configuration
<VirtualHost *:443>
    ServerName connections.example.com

    RewriteEngine On

    # Redirect hostname to Homepage
    RewriteRule ^\/$ https://connections.example.com/homepage [noescape,L,R]

    # Fix wrong wiki image URL
    RewriteRule "^/library/(.*)" "/wikis/form/api/library/$1" [R,L]

    SSLEnable
    SSLProtocolDisable SSLv2 SSLv3
</VirtualHost>
...

Adding EMPLOYEE_EXTENDED to all users

Today i read a question in the IBM Connections Forum about setting the EMPLOYEE_EXTENDED role to all users in a Connections deployment. It would be easy to set it directly in the database, but that’s not supported by IBM.

I wrote a little script some weeks ago, because i had the same request, but never published it. The good thing here it only uses supported commands.

So i use this question to add it as a new script to my GitHub Repository.

Here you see the source, it is simple a join of the tables emp_role_map and employee:

connect to peopledb;
EXPORT TO mail.txt OF DEL MODIFIED by NOCHARDEL
select e.PROF_MAIL FROM EMPINST.EMPLOYEE e
    inner join EMPINST.EMP_ROLE_MAP r
    on r.PROF_KEY=e.PROF_KEY
    where r.ROLE_ID!='employee.extended';
connect reset;

Just call it with db2 -tvf scriptname.sql.

The script exports a list of mail addresses of users without the specified role. This can then be used with the wsadmin command ProfilesService.setBatchRole(EMPLOYEE_EXTENDED, "mail.txt").

wsadmin.bat -lang jython -c 'ProfilesService.setBatchRole(EMPLOYEE_EXTENDED, "mail.txt")' as a oneliner will update the roles.

Scheduling the two commands with Windows Scheduler or cron is enough to update all users to the specified role.

Documentation is provided in the script head.