( automated ) testing of your osb services

A while ago i posted something about continuous integration for osb on a linux machine running bamboo. At the end of the build cycle (code checkin by dev, tool detects change, checkout, build, deploy) you like to run tests to see if the code that was just checked in…did not break anything.

In the case of service bus technology you preferably like to do end-to-end testing….meaning something like….
putting input somewhere….
service does its VETRO ( validate, enrich, transform,route and operate) thingie
and the ouput gets validated …

For synchronous services the strategy for testing could be…
- create a test facade that calls the service you like to test
- the service stores the output of the message using the wli osb reporting tables, storing $body witha key that has a unique id…for e.g. The internal $messageID variable…
- the service returns the messageID in the header with an inbound response to the testfacade
- thetestfacade does a query in the wli report data table using the id that was given in the header
I made a poc for that…and that works fine… If everything is synchronous …even over more services , this should enpose no problem…as long as the services pass the correct information via the header.. In my poc i had to do 2 tricz to get this running…
- include a timer that delays the query to the reporting table…the services repond quicker than the persistance of the body in the database…therefor i implemented a Thread.sleep in a java callout
- You need to sql the following in order to get the xml content from the blob table…
select utl_raw.cast_to_varchar2(dbms_lob.substr(BLOB_FIELD)) from TABLE_WITH_BLOB where ID = '';

…you can get the code from my repo….

For asynchronous services this is a complete different story… Often there is some queueing mechanism in place…and the main question will be then…
Which input file is responsible for which output file….with async services i cannot find a way to link the input with the output….perhaps content based?… Or do something smart with the facade ( test directory Or something like that) in any way …for now i will just assume that i am the only one testing or using a certain service at the time….then my strategy for now will be…
- let junit put file in the input of service
- wait some time
- let junit pick up the result from the configured endpoint of the service….do some testing…and voila

Select last record from oracle table

Just a mental note to myself….last record of OSB WLI reporting:

select * from wli_qs_report_attribute where msg_guid in(select max(msg_guid) from wli_qs_report_attribute);

or last N records

SELECT *
FROM (SELECT * FROM wli_qs_report_attribute)
WHERE ROWNUM < 10;

 

Accessing data from Oracle DB XML with OSB

Goal

Seeing Oracle XML database technology in action using the Oracle Service Bus.

Prerequisites

  • An Oracle Database that can handle tables of XMLTYPE, (Oracle DB XML): Tip download the excellent Virtual Box images from the http://www.oracle.com site
  • An Oracle Service Bus installation

Steps

  • Log in to the database (with sqlplus or sqldeveloper) and create a table of XML type;, for e.g.:
    create table chris_test of xmltype;
  • In sql developer you will see something like this:

wpid-Schermafbeelding2011-12-29om09.18.31-2011-12-29-09-10.png

NOTE that you did not have to create columns and their types…you just created a table of XMLtype :)

You can xquery this table…which I will show you later on…

Insert a record for e.g.

INSERT INTO chris_test
VALUES( XMLType('<?xml version="1.0"?>
<PO pono="1">
<PNAME>Chris_1</PNAME>
<CUSTNAME>Chris</CUSTNAME>
<SHIPADDR>
<STREET>43, Houtduif</STREET>
<CITY>Mijdrecht</CITY>
<STATE>Utrecht</STATE>
</SHIPADDR>
</PO>'));

you will see something like this then…

wpid-Schermafbeelding2011-12-29om10.07.10-2011-12-29-09-10.png

Now we will access this with OSB and xquery

  • For our example you need to create a datasource on your weblogic console (your OSB machine) named jdbcDS that can access the Oracle XML DB we just used
  • In your OEPE create a new XQuery resource, by select new –> XQuery Transformation (what is in a name)….

wpid-Schermafbeelding2011-12-29om11.25.28-2011-12-29-09-10.png

Put the following content in your xquery resource….

Note (the double ‘ signs)…escaping quotes

xquery version "1.0" encoding "UTF-8";
(:: pragma type="xs:anyType" ::)
declare namespace xf = "http://tempuri.org/OSBProject1/xquery/lookup/";
declare function xf:lookup()
as element(*) {
fn-bea:execute-sql(
'jdbcDs',
'rec',
'select XMLSERIALIZE(CONTENT COLUMN_VALUE AS CLOB INDENT SIZE=2)
from XMLTABLE
(
''for $i in fn:collection("oradb:/OE/CHRIS_TEST")/PO[CUSTNAME/text()=$REFERENCE]
return $i/CUSTNAME''
passing ''Chris'' as "REFERENCE"
)')
};

The fn-bea:execute-sql function takes 3 params in this case (other variations are possible…RTFM :)

  • 1st param is the name of the datasource
  • 2nd param is just a variablename for each record
  • 3rd param is the actual xquery / sql query …

Now just create a Proxy Service with an ASSIGN….that refers to this xquery….Happy testing…

wpid-Schermafbeelding2011-12-29om11.32.08-2011-12-29-09-10.png

        your proxy service…which will only output the results of the database xquery lookup

  • wpid-Schermafbeelding2011-12-29om11.33.16-2011-12-29-09-10.png

The Assign function…In which you refer to the XQuery resource…In my example it is called lookup.xq, I assigned this to variable XXX

        In the logging in do log the $XXX variabele….

        you will see something like this in your logging then…

<Dec 29, 2011 2:31:33 AM PST> <Error> <ALSB Logging> <BEA-000000> < [PipelinePairNode1, PipelinePairNode1_request, stage1, REQUEST] XXXXXXXXXXX =: <rec>

<_x0058_MLSERIALIZE_x0028_CONTENTCOLUMN_VALUEASCLOBINDENTSIZE_x00313_2_x0029_><CUSTNAME>Chris</CUSTNAME></_x0058_MLSERIALIZE_x0028_CONTENTCOLUMN_VALUEASCLOBINDENTSIZE_x00313_2_x0029_>

</rec>>

Continuous integration testing with Bamboo and OSB projects

Earlier I made some movies on this topic (lazy documentation)…now I have them on youtube…

Having experience with continuous integration tools like cruise control and hudson for java projects, I now wanted to implement this strategy for development and testing for an OSB/ ALSB project using Bamboo as CI tool.

My o my what a nightmare, nothing to be found on the internet about this topic…..

I finally got everything working eventually (3 days) …

GOAL

Have an environment that is deployed an test automatically after each change in the repository. One of the most important reasons for companies to implement Service Bus technology is transformation of messages. Therefore I like to achieve that after each change to the code, the Services are being tested on expected input and expected output. Ideally every OSB project will have a testproject that encapsulates the expectations and testcases for each project.

LINUX MACHINE RUNNING: Subversion/ Bamboo/ and OSB

Development Machine: Running whatever with Subversion plugin to checkin code

This picture below explains Continuous integration building and testing in a nutshell….

wpid-ci-2011-12-15-22-2011.jpg

Note: Actions 1 – 4 are performed on the build server, After a fix the sources gets checked in again…and the cycle starts again….HENCE continuos

PROBLEMS

I had several problems, but synchronisation was the main problem…

There are many blogs on the internet show you how to do an export and deployment of OSB projects…however if you have an OSBProject1 that is linked to an OSBConfiguration1 and you update the project (like an svn update, which is what BAMBOO does in case of a build)) then you have a problem…..

If you dont believe me try do the following:

  •  In OEPE create a OSBConfiguration
  • In OEPE create a OSBProject, put some test proxyService or business Services in them
  • In OEPE link the OSBProject to the OSBConfiguration
  • NOW exit eclipse and go to your workspace
  • make a tar of your current OSBProject (we will simulate SVN behaviour, by putting a backup back)
  • Open Eclipse and change something in your current current OSBProject, then Close eclipse again
  • Now untar your OSB backup …hereby overwriting the change in your current project
  • Now try do the export from OSB again…… (Remember you have your eclipse closed) you will notice that even tough you only have overwritten 1 proxy file that the build will fail because the file that you have overwritten is not synchroized with the OSBConfiguration anymore. For this you need to start eclipse again. I have spent a lot of time searching for the correlation and the only correlation I could find for now is between the Project and the Configarion in several files, no on artifact level tough. And no matter what I tried with manipulating (with shell sed scripts) nothing worked…appearantly there is some OSB/Eclipse magic going on….so in order to make this project run (Continuous Integration with OSB) I need to have OEPE/ eclipse open for the synchronization.

Lesson Learned: In case of Continous Integration whereby SVN is updating your OSBProject, have Eclipse/ OEPE run in the background to take care of the Synchronization otherwise the export of OSB which is done by com.bea.alsb.core.ConfigExport will fail with a 101 ERROR

After writing this POST I had another problem with projects with JCA components… When doing the export of the OSB projects I got the following error:

1-Dec-2011 09:28:51

21-Dec-2011 09:28:51

<Dec 21, 2011 9:28:51 AM CET> <Info> <ALSB Kernel> <BEA-397004> <Services for module “FLOW Resource” started.> 21-Dec-2011 09:28:59

Synchronizing “OSBConfiguration1″… 21-Dec-2011 09:29:02

<Dec 21, 2011 9:29:02 AM CET> <Error> <OSB Kernel> <BEA-380006> <No transport provider registered with ID: jca> 21-Dec-2011 09:29:03

com.bea.wli.config.component.ValidationException: Validation of BusinessService GM_PurcOrdeGoodRcpt/BusinessServices/PurcOrdeGoodRcpt_to_FFMW_File_BS : Diagnostics for BusinessService GM_PurcOrdeGoodRcpt/BusinessServices/PurcOrdeGoodRcpt_to_FFMW_File_BS 21-Dec-2011 09:29:03

21-Dec-2011 09:29:03

ERROR: <0> Transport provider with id ‘jca’ is not registered. failed 21-Dec-2011 09:29:03

at com.bea.wli.config.component.impl.ComponentTypeImpl.validateForCreate(ComponentTypeImpl.java:451) 21-Dec-2011 09:29:03

at com.bea.wli.config.component.impl.ComponentTypeImpl.createBare(ComponentTypeImpl.java:877) 21-Dec-2011 09:29:03

at com.bea.wli.config.component.impl.ComponentTypeImpl.create(ComponentTypeImpl.java:860) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.CreateResourceTask._execute(CreateResourceTask.java:97) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.SessionedTask.doExecute(SessionedTask.java:228) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.SessionedTask.doExecute(SessionedTask.java:191) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.CreateResourceTask.createResource(CreateResourceTask.java:82) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.tasks.CreateResourceTask.execute0(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.tasks.AbstractSynchronizerTask._execute(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.tasks.ExecutionTask._execute(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.SessionedTask$1.execute(SessionedTask.java:233) 21-Dec-2011 09:29:03

at com.bea.wli.config.transaction.TransactionalTask._doExecute(TransactionalTask.java:217) 21-Dec-2011 09:29:03

at com.bea.wli.config.transaction.TransactionalTask._doExecuteWithRetry(TransactionalTask.java:162) 21-Dec-2011 09:29:03

at com.bea.wli.config.transaction.TransactionalTask.doExecute(TransactionalTask.java:142) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.SessionedTask.doExecute(SessionedTask.java:236) 21-Dec-2011 09:29:03

at com.bea.wli.config.task.impl.SessionedTask.doExecute(SessionedTask.java:191) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.AbstractSynchronizer.executeEx(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.AbstractSynchronizer.execute(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.AbstractFullSynchronizer.run(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.config.synchronize.RefreshContainerSynchronizer.run(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.repositories.ConfigExportApplication.synchronize(Unknown Source) 21-Dec-2011 09:29:03

at com.bea.alsb.core.internal.repositories.ConfigExportApplication.start(Unknown Source) 21-Dec-2011 09:29:03

at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) 21-Dec-2011 09:29:03

at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) 21-Dec-2011 09:29:03

at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) 21-Dec-2011 09:29:03

at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369) 21-Dec-2011 09:29:03

at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179) 21-Dec-2011 09:29:03

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 21-Dec-2011 09:29:03

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 21-Dec-2011 09:29:03

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 21-Dec-2011 09:29:03

at java.lang.reflect.Method.invoke(Method.java:597) 21-Dec-2011 09:29:03

at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:620) 21-Dec-2011 09:29:03

at org.eclipse.equinox.launcher.Main.basicRun(Main.java:575) 21-Dec-2011 09:29:03

at org.eclipse.equinox.launcher.Main.run(Main.java:1408) 21-Dec-2011 09:29:03

at org.eclipse.equinox.launcher.Main.main(Main.java:1384) 21-Dec-2011 09:29:03

> 21-Dec-2011 09:29:03

Exporting to /tmp/sbconfig.jar… 21-Dec-2011 09:29:03

Error during configuration jar export.

Until now everything was working with with Standard OSB artifiacts, however when Projects refer to JCA components you go the error like above.

After some googling aroung I found out that it was related to the 64 bit version of OEPE that I was using for creating the Export. I used the same scripts on the same projects on my local Linux machine which runs a 32 bit version of OEPE and then it works fine….how about that.

STEPS

  • On your build server (see pic) make sure you have OSB with OEPE installed
  • Run Eclipse OEPE on your linux machine; You need to run Eclipse on the Linux machine, I will explain how to do this (generate key, X11 forwarding etc etc) (some tips, run eclipse by nohup ./eclipse & and make sure your session is not killed by updates and a rebooting machine…happened to me too a couple of times)
  • On your build server (see pic) make sure you have BAMBOO install and configured
  • Eclipse steps
  • Configure Bamboo

==================================================================

1.        INSTALL OEPE (choose 32 bits (for now))

This document refers to the installation steps…

2.        Run Eclipse (OEPE) on your linux machine

You will probably have the following situation, you work on windows machines, you access your linux machine using putty or some kind of other terminal and you have a personal username and password to access the machine.

Once you accessed the machine, you will need to sudo in order to become the user for starting and stopping your oracle/ bea software.

Note It is not possible to open an X11 session with X11 forwarding while your logged in as user A and you start the X11 session as user B… the X11 forwarding needs the .Xauthority file which resides in your home directory of your user.

The following steps will explain you how to start an X11 session on your windows box….the X11 session is needed for the Eclipse GUI on linux.

        - Logon to the linux box using your own account, then sudo to the user you suppose to be… for example sudo su – osb

        - Create a ssh key with the following command (I prefer type DSA…not mandatory) ssh-keygen -t dsa

wpid-Schermafbeelding2011-12-25om15.58.30-2011-12-15-22-2014.png

        just fill in the default values…do not fill in a password…this is much easier :)

        At the end you will have a .ssh directory in your home directory with the following files:

wpid-Schermafbeelding2011-12-25om16.25.51-2011-12-15-22-2014.png

        

  • now copy the id_dsa.pub into a file named authorized_keys, this file is not created by default
  • now copy the private key (id_dsa) to your windows machine using winscp or some other SFTP client.
  • Download putty with puttygen included from one of the many sites for e.g. http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
  • Have Xming installed, this makes your X11 gui visible in a windows frame, download it from here http://sourceforge.net/projects/xming/
  • Start your Xming server
  • On your windows box convert your private key into the PKK format by running….puttygen.exe

wpid-Schermafbeelding2011-12-27om20.46.31-2011-12-15-22-2014.png

  • press on Load Private Key and load the private key you just uploaded
  • After succesful loading you will see something like the screen below:

        wpid-Schermafbeelding2011-12-27om20.51.59-2011-12-15-22-2014.png

  •  Then Press the “Save Private Key” button in order to save the key in ppk format…save it as <filename>.ppk (Note in the example above I used a password…this is not mandatory) PPK is Putty’s own format…
  • This key you can use for logging with your user…If it is not working …talk to your NIX administrator…it could be that some permissions needs to be set on /etc/ssh/sshd_config
  • In your putty session configure your putty session to use the PPK key you just created…see screen below:

wpid-Schermafbeelding2011-12-27om21.33.41-2011-12-15-22-2014.png

  • Also turn on your X11 forwarding…see screen below…

wpid-Schermafbeelding2011-12-27om21.44.53-2011-12-15-22-2014.png

  •  The MIT Magic Cookie will be saved in your.XAuthority file in your home directory…when you log in….Instructions for setting up X11 on your Linux box…for e.g. http://www.wikihow.com/Configure-X11-in-Linux
  • Now start eclipse by running nohup ./eclipse & (this will make the oepe run even after closing down the session (unless you kill eclipse itself)…also make sure that your linus administrator did not configure a session timeout for your sshd sessions

3.        BAMBOO install

Bamboo installation via this guide

4.        Eclipse Steps        

  • create an OSB configurationIn your OEPE project explorer, right click the project browser and select OSB create Osb Configuration, Name it something without spaces, for e.g. OSBConfiguration1
  • via console checkout all the projects you want to have build…

        This is something you do only initially: Open a console and check out your projects in the workspace of eclipse

        for e.g. cd /home/oracle/workspace

        svn co http://someServer.com/svn/OSBProject1

wpid-Schermafbeelding2011-12-28om00.35.04-2011-12-15-22-2010.png

- then in eclipse import your project into eclipse…Right click resources explorer, IMPORT , IMPORT, Existing projects into workspace

wpid-Schermafbeelding2011-12-28om00.44.01-2011-12-15-22-2010.png

wpid-Schermafbeelding2011-12-28om00.45.09-2011-12-15-22-2010.png

Select the project you just checked out, you get something like this below then…

wpid-Schermafbeelding2011-12-28om00.46.44-2011-12-15-22-2010.png

  • link your projects by dragging the project into the OSBConfiguration, it will look something like this then…

wpid-Schermafbeelding2011-12-28om00.50.47-2011-12-15-22-2010.png

4 Configure Bamboo

Put the build scripts in a directory on your server….

for e.g. in /home/oracle/scripts/osbbuild we will need the following scripts:

import.py; a python script for actually deploying the osb artifacts into a preconfigured OSB environment

build.xml; the ant script responsible for building and copying stuff

build.properties; the file needed for configuring the scripts

Build.properties

================

importJar=/tmp/sbconfig.jar
fmw.home=/opt/oracle/fmw/
wls.username=weblogic
wls.password=weblogic1
wls.server=t3://server.com:7001
config.project="OSBConfiguration1"
config.jar=/tmp/sbconfig.jar
config.includeDependencies=true
workspace.dir=/home/soa/workspace

Build.xml

================

Note: In order to run the sbconfig.jar export command line use the java command like below (it is a good alternative for people who do

not like to wrap code into XML)

commandLine

java -Dosb.home=/opt/oracle/fmw11_1_1_5-OEPE/osb/ -Xms384m -Xmx768m -Dweblogic.home=/opt/oracle/fmw/wlserver_10.3/ -jar /opt/oracle/fmw11_1_1_5-OEPE/oepe/plugins/org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar -data /home/soa/workspace -application com.bea.alsb.core.ConfigExport -configProject OSBConfiguration1 -configJar /tmp/sbconfig.jar

ant

<?xml version="1.0" encoding="windows-1252" ?>
<project name="ConfigExport">
   <taskdef name="wlst" classname="weblogic.ant.taskdefs.management.WLSTTask"/>
   <property file="/home/soa/software/BUILD_SCRIPT/build.properties"/>
<property name="eclipse.home" value="/opt/oracle/fmw11_1_1_5-OEPE/oepe"/>
<property name="weblogic.home" value="${fmw.home}/wlserver_10.3"/>
<property name="metadata.dir" value="${workspace.dir}/.metadata"/>
<property name="osb.home" value="${fmw.home}/Oracle_OSB1"/>
<property name="eclipse.build.jar" value="${eclipse.home}/plugins/org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar"/>

<target name="buildOsb" depends="copyAndClean">
<java dir="${eclipse.home}" jar="${eclipse.build.jar}" fork="true" failonerror="true" maxmemory="768m">
<jvmarg line="-XX:MaxPermSize=256m"/>
<arg line="-data ${workspace.dir}"/>
<arg line="-application com.bea.alsb.core.ConfigExport"/>
<arg line="-configProject ${config.project}"/>
<arg line="-configJar ${config.jar}"/>
<arg line="-includeDependencies ${config.includeDependencies}"/>
<sysproperty key="weblogic.home" value="${weblogic.home}"/>
<sysproperty key="osb.home" value="${osb.home}"/>
<sysproperty key="osgi.bundlefile.limit" value="500"/>
<sysproperty key="harvester.home" value="${osb.home}/harvester"/>
<sysproperty key="osgi.nl" value="en_US"/>
<sysproperty key="sun.lang.ClassLoader.allowArraySyntax" value="true"/>
 </java>
 </target>
 <target name="copyAndClean">
 <delete>
<fileset dir="/tmp" includes="sbconfig.jar"/>
 </delete>
 <copy todir="${workspace.dir}/${ProjectName}/">
 <fileset dir="/opt/oracle/bamboo-home/xml-data/build-dir/${BambooName}"/>
 </copy>
 </target>

 <target name="importOsb">
<java classname="weblogic.WLST" fork="true">
<arg line="import.py build.properties"/>
</java>
</target>
</project>

Import.py

================


Import.py

from java.util import HashMap
 from java.util import HashSet
 from java.util import ArrayList
 from java.io import FileInputStream

from com.bea.wli.sb.util import Refs
 from com.bea.wli.config.customization import Customization
 from com.bea.wli.sb.management.importexport import ALSBImportOperation

import sys
 import datetime

#=======================================================================================
 # Entry function to deploy project configuration and resources
 # into a ALSB domain
 #=======================================================================================

def importToALSBDomain(importConfigFile):
 try:
 SessionMBean = None
 print 'Loading Deployment config from :', importConfigFile
 exportConfigProp = loadProps(importConfigFile)
 adminUrl = exportConfigProp.get("wls.server")
 importUser = exportConfigProp.get("wls.username")
 importPassword = exportConfigProp.get("wls.password")

importJar = exportConfigProp.get("importJar")
 customFile = exportConfigProp.get("customizationFile")

passphrase = exportConfigProp.get("wls.password")
 project = exportConfigProp.get("config.subprojects")

connectToServer(importUser, importPassword, adminUrl)

print 'Attempting to import :', importJar, "on ALSB Admin Server listening on :", adminUrl

theBytes = readBinaryFile(importJar)
 print 'Read file', importJar
 sessionName = createSessionName()
 print 'Created session', sessionName
 SessionMBean = getSessionManagementMBean(sessionName)
 print 'SessionMBean started session'
 ALSBConfigurationMBean = findService(String("ALSBConfiguration.").concat(sessionName), "com.bea.wli.sb.management.configuration.ALSBConfigurationMBean")
 print "ALSBConfiguration MBean found", ALSBConfigurationMBean
 ALSBConfigurationMBean.uploadJarFile(theBytes)
 print 'Jar Uploaded'

if project == None:
 print 'No project specified, additive deployment performed'
 alsbJarInfo = ALSBConfigurationMBean.getImportJarInfo()
 alsbImportPlan = alsbJarInfo.getDefaultImportPlan()
 alsbImportPlan.setPassphrase(passphrase)
 alsbImportPlan.setPreserveExistingEnvValues(true)
 importResult = ALSBConfigurationMBean.importUploaded(alsbImportPlan)
 SessionMBean.activateSession(sessionName, "Complete test import with customization using wlst")
 else:
 print 'ALSB project', project, 'will get overlaid'
 alsbJarInfo = ALSBConfigurationMBean.getImportJarInfo()
 alsbImportPlan = alsbJarInfo.getDefaultImportPlan()
 alsbImportPlan.setPassphrase(passphrase)
 operationMap=HashMap()
 operationMap = alsbImportPlan.getOperations()
 print
 print 'Default importPlan'
 printOpMap(operationMap)
 set = operationMap.entrySet()

alsbImportPlan.setPreserveExistingEnvValues(true)

#boolean
 abort = false
 #list of created ref
 createdRef = ArrayList()

for entry in set:
 ref = entry.getKey()
 op = entry.getValue()
 #set different logic based on the resource type
 type = ref.getTypeId
 if type == Refs.SERVICE_ACCOUNT_TYPE or type == Refs.SERVICE_PROVIDER_TYPE:
 if op.getOperation() == ALSBImportOperation.Operation.Create:
 print 'Unable to import a service account or a service provider on a target system', ref
 abort = true
 elif op.getOperation() == ALSBImportOperation.Operation.Create:
 #keep the list of created resources
 createdRef.add(ref)

if abort == true :
 print 'This jar must be imported manually to resolve the service account and service provider dependencies'
 SessionMBean.discardSession(sessionName)
 raise

print
 print 'Modified importPlan'
 printOpMap(operationMap)
 importResult = ALSBConfigurationMBean.importUploaded(alsbImportPlan)

printDiagMap(importResult.getImportDiagnostics())

if importResult.getFailed().isEmpty() == false:
 print 'One or more resources could not be imported properly'
 raise
 print
 print 'Modified importPlan'
 printOpMap(operationMap)
 importResult = ALSBConfigurationMBean.importUploaded(alsbImportPlan)

printDiagMap(importResult.getImportDiagnostics())

if importResult.getFailed().isEmpty() == false:
 print 'One or more resources could not be imported properly'
 raise

#customize if a customization file is specified
 #affects only the created resources
 if customFile != None :
 print 'Loading customization File', customFile
 print 'Customization applied to the created resources only', createdRef
 iStream = FileInputStream(customFile)
 customizationList = Customization.fromXML(iStream)
 filteredCustomizationList = ArrayList()
 setRef = HashSet(createdRef)
 # apply a filter to all the customizations to narrow the target to the created resources
 for customization in customizationList:
 print customization
 newcustomization = customization.clone(setRef)
 filteredCustomizationList.add(newcustomization)

ALSBConfigurationMBean.customize(filteredCustomizationList)

SessionMBean.activateSession(sessionName, "Complete test import with customization using wlst")

print "Deployment of : " + importJar + " successful"
 except:
 print "Unexpected error:", sys.exc_info()[0]
 if SessionMBean != None:
 SessionMBean.discardSession(sessionName)
 raise

#=======================================================================================
 # Utility function to print the list of operations
 #=======================================================================================
 def printOpMap(map):
 set = map.entrySet()
 for entry in set:
 op = entry.getValue()
 print op.getOperation(),
 ref = entry.getKey()
 print ref
 print

#=======================================================================================
 # Utility function to print the diagnostics
 #=======================================================================================
 def printDiagMap(map):
 set = map.entrySet()
 for entry in set:
 diag = entry.getValue().toString()
 print

#=======================================================================================
 # Utility function to load properties from a config file
 #=======================================================================================

def loadProps(configPropFile):
 propInputStream = FileInputStream(configPropFile)
 configProps = Properties()
 configProps.load(propInputStream)
 return configProps

#=======================================================================================
 # Connect to the Admin Server
 #=======================================================================================

def connectToServer(username, password, url):
 connect(username, password, url)
 domainRuntime()

#=======================================================================================
 # Utility function to read a binary file
 #=======================================================================================
 def readBinaryFile(fileName):
 file = open(fileName, 'rb')
 bytes = file.read()
 return bytes

#=======================================================================================
 # Utility function to create an arbitrary session name
 #=======================================================================================
 def createSessionName():
 currentTime = datetime.datetime.utcnow()
 sessionName = String("BAMBOO_CI_BUILD_"+str(currentTime.day)+"_"+str(currentTime.month)+"_"+str(System.currentTimeMillis()))
 return sessionName

#=======================================================================================
 # Utility function to load a session MBeans
 #=======================================================================================
 def getSessionManagementMBean(sessionName):
 SessionMBean = findService("SessionManagement", "com.bea.wli.sb.management.configuration.SessionManagementMBean")
 SessionMBean.createSession(sessionName)
 return SessionMBean

# IMPORT script init
 try:
 # import the service bus configuration
 # argv[1] is the export config properties file
 importToALSBDomain(sys.argv[1])

except:
 print "Unexpected error: ", sys.exc_info()[0]
 dumpStack()
 raise

===========================

Now we configure Bamboo…Login to the bamboo console

wpid-bamboo1-2011-12-15-22-2010.png

Create a Plan…which will encorporate the behaviour of 1 or more projects….than add a project and configure its tasks

the 1st task is SVN Checkout…see the1st picture (make sure you checkout from the proper trunk/ branch or whatever)

2nd and 3rd taks are ANT tasks, 1st ANT task is for building/ exporting the osb artifacts (with the changes from the previous subversion checkout), 2nd ANT task will call the actual import task for importing the deployable onto the preconfigured deployment server….You could also combine this into 1 task, but I chose to seperate it into 2 tasks for debugging purposes

The Maven task is needed for executing the Junit test cases (will be handled another time)…

The last step for parsing the sure-fire test results is not really needed…and will not be discussed any further

wpid-Bamboo2-2011-12-15-22-2010.png

in the 1st ANT task I configure the following in Bamboo

- Task description; just a name for the task

- Excecutable; a link to the preconfigured ant installation which must reside on the same machine

- build file: this is the location to the build.xml file (command line it would be something like ant -f <build.xml>)

- In Target I fill in the following

        - 1 st param is the name of the task: buildOsb (see source)

        - 2nd param -DProjectName=CommonOSB; in the build.xml you see a reference to this param ${ProjectName} this is name of the project we just checked out from SVN, this reference is also used in the eclipse workspace

        - 3rd param -DBambooName=<someBambooName>; in the build.xml you see a reference to this param ${BambooName} this is the Bamboo directory we are using, this is needed because we will copy files from this directory into our eclipse workspace

        - in CLASSPATH, I just used the $CLASSPATH which I get if I do a setWLSEnv.sh

Best of luck to you…may the force be with you :)

If you need help, just give me a mail: chris@datalinks.nl

OSB java callout and presenting ProxyService output as HTML

This is what I lke to achieve:

Implement a simple ProxyService that reads a file with a Java CallOut,

do something with the File and present it’s output in HTML

I know ProxyServices are note meant for Presentational stuff and all that other blabla…,

but I like to use technology the way I like it and not because people say it needs to be used in a certain way.

In Eclipse:

  • create OSBConfiguration
  • create OSBProject
  • create Java Project

In the java project create a class that does your file Reading and creates HTML output.

something like this…. NOTE that your output method needs to be STATIC

import java.io.File;
public class VersionReader {

public VersionReader(){
}

public static String constructInfo(){
StringBuffer sb = new StringBuffer();
sb.append("<HTML>");
sb.append("<H1> HELLO WORLD !!!!!! </H1>");
sb.append("</HTML>");
}
}

! OSB can directly handle (do something with it) SimpleTypes and XMLTypes, ComplexTypes like Lists of of Objects or HashMaps can only be passed through as reference.

In eclipse create an export of your java sources in the form of a JAR file:

wpid-wpid-Schermafbeelding2011-12-01om05.55.08-2011-12-1-05-37-2011-12-1-05-37.png

Right Click on your Java Project do a export, select OTHER and then choose JAR

In your OSB Project Create a Jar Folder and Drag the JAR file you just created into your OSBProjects’ JAR folder….

wpid-wpid-Schermafbeelding2011-12-01om06.10.52-2011-12-1-05-37-2011-12-1-05-37.png

Now in your OSB Project Create your Proxy Service

Click something together :) That looks something like this

PipeLine PairNode, 2 stages, stage 1 java call Out, stage 2 replace and transport header

Defined 2 stages for debugging purposes…not really needed

wpid-wpid-Schermafbeelding2011-12-01om07.51.47-2011-12-1-05-37-2011-12-1-05-37.png

Confgure your Java callout like this…

wpid-wpid-Schermafbeelding2011-12-01om07.55.28-2011-12-1-05-37-2011-12-1-05-37.png

In the Java callout you will get a dialogue that forces you to point to a java class (point to the JAR file you dragged earlier)

then select the static Method you programmed earlier.

As you will see, OSB is smart enough to see that the result Type is of the Simple Type String. The output I assing to an OSB variable named htmlOutput

In the replace Message Processing step I configure the following:

wpid-wpid-Schermafbeelding2011-12-01om07.59.08-2011-12-1-05-37-2011-12-1-05-37.png

Simply put I place the content of the Java Callout into the body result of this ProxyService, Note that I checked the Replace Node Contents radio button

So far this has been standard OSB stuff, but now the trick for Presenting XML output (that contains a HTML string) in your browser….

You need to Configure the COMMUNICATION step Transport header in order to Trick your browser in order to accept this…

wpid-wpid-Schermafbeelding2011-12-01om08.05.46-2011-12-1-05-37-2011-12-1-05-37.png

Direction is INBOUND RESPONSE

Select CONTENT-TYPE and configure the expresson to “text/html”

Deploy this stuff and call the Proxy Service in your browser and behold your result:

wpid-wpid-Schermafbeelding2011-12-01om08.15.47-2011-12-1-05-37-2011-12-1-05-37.png

1st script for version control OSB artifacts

#########################################################################
#
#    Name:         addVersion.sh
#    Version:      0.1; Initial version
#    Description:  This adds a version Number to your OSB artificats, you will
#                                  you will see the versionnumber in the description field (in sbconsole)
#                                  it will also maintain a version file in the home directory of the soa user
#                  Note: Dirty hack in order to make sed accept params…
#                  Note that $1 must be the version Name then!!!
#                  Versioning sources to v0.3  addVersion.sh v0.3
#    Date:         24 november 2011
#    Author:       Chris Vugrinec (chris@datalinks.nl)
#
#########################################################################

reposistoryLocation=”https://XXX.com/svn/”
# Check pre requisites
if [ $# -ne 2 ];
then
clear
echo “Usage: $0 <projectName in svn> <versionTag>”
echo “choose one of these projects to tag:”
echo “in case of a lang name with spaces, use doublequotes”
echo “===================================================”
svn list $reposistoryLocation | while read LINE
do
echo – $LINE | sed -e ‘s/\///g’
done
exit 127
fi

repoName=`echo $1 | sed -e ‘s/ /\ /g’`
repoNameMod=`echo $1 | sed -e ‘s/ /_/g’`
version=$2

# checkout from repo
mkdir $repoName
cd $repoName
svn co $reposistoryLocation”$repoName”/trunk
cd trunk

# Iterate through project files
for file in `find . -type f | grep -v $0 | grep -v .svn`
do
# Adding version to all Services
if [[ `grep "<ser:coreEntry isProxy" $file` != '' ]]
then
cat $file | sed ‘
/<ser:coreEntry isProxy.*$/ a\
<ser:description>’”$2″‘</ser:description>
‘ > $file.tmp
mv $file.tmp $file
fi

# Adding version to all WSDL
if [[ `grep "<con:wsdlEntry xmlns:con" $file` != '' ]]
then
cat $file | sed ‘
/<con:wsdlEntry xmlns:con.*$/ a\
<con:description>’”$2″‘</con:description>
‘ > $file.tmp
mv $file.tmp $file
fi

# Adding version to all JCA
if [[ `grep "<con:jcaEntry xmlns" $file` != '' ]]
then
cat $file | sed ‘
/<con:jcaEntry xmlns.*$/ a\
<con:description>’”$2″‘</con:description>
‘ > $file.tmp
mv $file.tmp $file
fi

# Adding version to all XSD
if [[ `grep "<con:schemaEntry xmlns" $file` != '' ]]
then
cat $file | sed ‘
/<con:schemaEntry xmlns.*$/ a\
<con:description>’”$2″‘</con:description>
‘ > $file.tmp
mv $file.tmp $file
fi
done

svn mkdir  $reposistoryLocation$repoName/tags/$version -m “Tagging version $version with $0″
svn ci -m “Tagging version $version with $0″
svn copy $reposistoryLocation$repoName/trunk $reposistoryLocation$repoName/tags/$version -m “Tagging version $version with $0″

# create the deployable
# cleanup
#find ./ -name .settings -exec rm -rf {} \; 2>/dev/null
#find ./ -name .svn -exec rm -rf {} \; 2>/dev/null
cd -
#jar -cvfM $repoNameMod-$2.jar $repoName

 

 

Ldap browser

Note to myself…

a great ldap browser I used: apache directory studio

http://directory.apache.org/studio/download/download-windows.html

Versioning OSB artifacts

Versioning of OSB artificats is a bit more complex…than regular web applications..

Some people use the version numbers in the artifacts which is a lot of hassle.

Why do we do versioning: we like to see which version is deployed in production (or any other environment)

my strategy

Tagging version

  • svn checkout
  • create versionFile (this will be used if you deploy for the version webapp)
  • add version number to all artifacts
  • svn mkdir tags/<version nr>
  • svn checkin
  • svn copy trunk/project tags/<version nr>

Making Description empty again

  • svn checkout from trunk
  • remove versionFile
  • remove version from artifacts
  • checkin

Deployment based on tag

  • checkout tag from svn
  • make sbconfig.jar
  • generate list of artifacts
  • publish generated artifact list with version number to versionProxy

Checkin version

  • Via the web app…will read textfile or via versionProxyService

Decided to make a app for it…..TO BE continued!!!!

Things I tried adding version number to description

  • Deployment plan, does not work for OSB deployments
  • Customization file, only search and replace of things it can find…not descriptions :(
  • wlst, not possible to change description of items…you can enable disable and rename but no property for description anywhere
  • java, not possible to change description of items…you can enable disable and rename but no property for description anywhere, the ProxyService? BusinessService and ALSBConfigurationMBean do not support change of description :(

weblogic performance issue with urandom on Linux

I remember I had this issue a while age….but to my shock I did not blog it…Because my great inspirator Mr Vernetto Blogs everything with javamonamour…I will be a better boy now and start documenting…

Anyway

Because weblogic uses some libraries of the OS for encryption and decryption it is adviced to use the /dev/urandom library in stead of the defaul /dev/random lib…this can have a major impact on performance.
there are several ways to enforce this…but my favorits is to do it on OS level by making a symbolic link.

as root:
cd /dev
rm -f random
ln -s /dev/urandom random

here is another link:

http://weblogic-wonders.com/weblogic/2010/11/10/issues-relating-to-urandomrandom-on-weblogic-server-in-linux-environment/

Setup subversion repository

not going to explain that you need the proper rights, how to install subversion…just the setup…on a linux box of course :)

svnadmin create /home/oracle/repository

vi /home/oracle/repository/conf/svnserve.conf

add the following lines

anon-access = none

auth-access = write

password-db = passwdFile

vi /home/oracle/repository/conf/passwdFile

put here something like this

someUserName = somePassword

In your shell define the default editor for SVN, else you might get a message like this:

[oracle@soabpm-vm base_osb_domain]$ svn mkdir file://home/oracle/repository/myFirstSvnProject

svn: Could not use external editor to fetch log message; consider setting the $SVN_EDITOR environment variable or using the –message (-m) or –file (-F) options

svn: None of the environment variables SVN_EDITOR, VISUAL or EDITOR is set, and no ‘editor-cmd’ run-time configuration option was found

you can set vi as your default SVN editor like this:

[oracle@soabpm-vm base_osb_domain]$ export SVN_EDITOR=vi

svn mkdir file://home/oracle/repository/myFirstSvnProject

lets put some stuff there…

touch welcomeToMyBlogg

svn import welcomeToMyBlogg file:///home/oracle/repository/myFirstSvnProject

Notice that you have to put /// slashes !!!

To work under version control the best thing to do is the following:

- mv the current project directory to a tmp directory (backup precaution)

- Checkout the project from the repository… and work with this copy, you can checkout via:

svn co file:///home/oracle/repository/myFirstSvnProject

NOTE

it is best practice using the promotional model (google it ;)

go to your project directory

svn mkdir trunk

svn mkdir tags

svn mkdir branches

(trunk will be the main (latest) version where developers work on, tags are meant for tagging certain releases (usually this is a copy of certain files in the trunk at a certain moment), branches are meant for developers who need to go their own way ;) )

start the deamon

svnserve -d

(for always starting after reboot do: chkconfig svnserve on)

you can check if your repo works with

svn co svn+ssh://username@hostname/path to repo

while we are at it…

Here is how to tag a version 0.1

(do not do a svn create tags/0.1)

svn copy file:///home/oracle/repository/Project1/trunk file:///home/oracle/repositoryProject1/tags/0.1 -m “version 0.1″

check the contents of this release with the following command

svn list file:///home/oracle/repository/Project1/tags/0.1

in order to make your repository availabel via http apache….enter the following config to your apache

LoadModule dav_svn_module modules/mod_dav_svn.so

LoadModule authz_svn_module modules/mod_authz_svn.so

<Location /repo>

DAV svn

SVNPath home/oracle/repository/

AuthzSVNAccessFile /etc/httpd/passwd/svn_authz.conf

AuthType Basic

AuthName “Datalinks Subversion Repository”

#make this file with the htpasswd command

AuthUserFile /.htacces

Require valid-user

</Location>