Enhanced Trace

• Introduction
• User Guide
• Installation
• History & Limitations
• Roadmap

Introduction

Tracing IDM drivers can be difficult, especially if you have a lot of policies. Designer has a colored trace view, but mixes all drivers. The log files on the server are missing the nice colors.

This IDM designer plugin tries to make tracing IDM drivers simpler. It provides the following features:

• Simple context menu to start the trace.
• Parse the IDM trace and present an outline tree view.
• Filter the trace per driver.
• Drill down on an event with a fish-bone overview.

NetIQ community members' rating: (5 stars, 2 votes, average 5 out of 5)

This software is MPL 2.0

User Guide

Starting the trace

Note: The Enhanced Trace plugin is optimized for trace level 3.

The trace is server specific. It uses an extended LDAP operation to a specific server to get the trace data directly from eDirectory running on that given server. Right click on the server object (in the outline view) running your driver, select Live -> Trace. This will open and start the enhanced trace on the selected server.

Start Enhanced trace

Note that the trace will start using the properties defined on the given server object. If no information is found for eg dns or IP address information, it will fall back to the IDVault properties. So, on a multi-server environment, make sure to complete the DNS or host address name of the server!

When you start the trace, you will get a screen similar to the standard trace screen.
Enhanced trace live editor view

Outline Tree view

As soon as the trace starts, it will parse the received trace messages and present a tree view in the outline pane. Each node represents a trace step. The plugin tries to group trace messages, not only per driver and event, but also create a hierarchy of trace messages.

The top level event gives a short summary: type of event, object class, object cn, association and worst status in the event.

The highlighted event below is for a publisher event of the JDBC driver 'Corner'. It shows an add event of the USER table, with cn (in JDBC terms the primary key of the table) equal to 33300.
Enhanced trace tree view

Filter trace

You can filter the traces to a given driver. Note: the names are not the driver names, but the trace names as defined on the driver. If not specified, this defaults to the driver name.

Drill down through the events

You can drill down through the events and zoom in. Whatever event you click on, you will always drill down just one level.

Note that this is a drill down feature: eg: on a high level, every command performed in eg the command transformation policy, will be traced as being part of the command transformation policy, even if that command passes the output transformation. If you drill down on the subcommand, it will be split up into it's individual components and thus have trace messages in the output transformation.

Outline view

The outline view of the drilled down event shows the fish-bone. The blocks are colored if an trace message for that block is found.
Drill down fishbone view

Quick navigation options:

• Clicking on a fish-bone block jumps to the first event of that block.
• Context menus on the blocks allow you to quickly jump to:
  • a specific policy. All policies that contain a trace message are listed.
  • a status message. All status messages traced within the policy set are listed.
  • a trace message. All trace messages traced within the policy set are listed.
• The overlay icons indicate the worst status message in a given block. Clicking on the overlay icon will jump to that status message. In the screenshot, both the publisher creation and association processor show a 'warning' status icon. In this example, clicking on the icon in the association processor would reveal unresolvable association references, while the creation policy would reveal a veto of the event.

Editor view

The editor view displays the actual trace messages. You can copy the single message or message tree, message details (XDS document) or both. Double clicking a message will drill down one level (if possible).
Enhanced trace detail editor view

The editor view contains both a refresh and rescan button:

• refresh will refresh and reanalyze the current event. This is in case you opened the detailed view before the event completed.
• rescan will search for the latest event in the live trace with the same signature as the current event (experimental).

Installation

I am in the process of moving all plugins and extensions to github, including an IDM update site. This extension is the first one available via the IDM/eclipse update feature. Unfortunately, NetIQ removed the standard 'install software' feature from the eclipse platform. Luckily, they included a package update site feature that works more or less the same.

In order to install this plugin, add the following site as a package update site to IDM:

I have read and accept the MPL (Mozilla Public License) 2.0. license (click checkbox to view the URL).

https://raw.githubusercontent.com/scauwe/IDMSite/master/

and try to update you IDM packages. This will now find this plugin. Select it and perform the update.

FYI: Adding an update site:

Window > Preferences > NetQ > Package Manager > Online Updates 

Add a new update URL as defined above.

History

July 2015, version 0.1.0

• Initial release
• Known limitations:
  • When running this plugin in a server with lots of drivers (eg >20), and each driver is tracing at level 3, the plugin can become a performance bottleneck, stressing Designer. Potential solution: apply the tree-view filter to the editor view as well (or: move the filter from the tree to the editor view).
  • Initial opening of the enhanced trace editor is slow.
  • Bigger XDS messages can get truncated in the trace. Under investigation. This is possible caused by the fact that these trace messages are broken down by eDirectory into several partial events, without any method to correlate between the two (or more) parts. The plugin is thus unable to find back the 'parent' event of the current partial event.
  • Only tested in Designer 4.0 and 4.5 on Windows and Linux. If you are running something else, feel free to provide feedback.
  • The label of the filter policy set (both on subscriber as on publisher) in the outline view is not visible if no trace messages are found for the given filter.

Roadmap

I'd love to add the following features:

• Save and open the traces.
• Search in the events.
• Option to directly open and focus the policy editor from a corresponding trace message (I am dreaming, I know). I think that this one will require changes to NetIQ's (undocumented) designer data model.

Association Editor

• Introduction
• Installation
• User Guide
• History
• Known Limitations

Introduction

You all know the DAModifier tool? Nice tool, but it depends on the Novell Client and Windows OS. This designer plug-in allows you to do similar actions: import, modify or export IDM driver associations, but runs on any system Designer runs on.

NetIQ community members' rating: (5 stars, 2 votes, average 5 out of 5)

This software is MPL 2.0

Installation

I am in the process of migrating all my plugins to github, including an automated update site. This plugin is however not yet migrated. Download is available from Cool solutions. Cool solutions does not allow upload of jars, so rename the attached zip to .jar and copy the jar to the plugins folder of your designer and restart designer if already running. I tested this with Designer 4.0.1 and 4.0.2 in Linux and Windows.Version 0.4 supports Designer 4.6 LDAP and non-LDAP.

User Guide

Open Dialogue

Right click on a driver object in the outline view or the driver connection in the modeler view and select Live ? Association Editor or right click on the application (in modeler or outline view) and select Driver > Live > Association Editor.
New menu entry: Association Editor

A dialogue will pop-up where you can select what you want to do.

Export associations.

Export associations will export the associations of the selected driver to a tab separated file. This file will contain the object DN (ldap format), the associations state and the association value. Enter the required information (search base, ldap filter, association status and the target file) and press Start.
Export associations dialog

Modify associations

This allows modification of the association status. Enter the required information (search base, ldap filter, 'from' association status, 'to' association status) and press Start. Optionally, a log file is generated. This log file (tab separated) contains the object DN, the old state, the association value and the action taken.
Modify associations dialog

Import associations

This allows for importing associations. Enter the required information (file to import) and press Start. Optionally, you can limit the import to a 'test only'. This will generate a log file of the actions it would take. The file to import is a tab separated file with the following columns: object dn (ldap), association state, association value (same format as the export option). Optionally (unless 'validate only' is checked) a log file is generated. This log file (tab separated) is the same as the input file, but adds an action column containing the action taken (or that would be taken in case of 'validate only'). Note: when 'validate only' is not selected, importing will always delete the current associations for the object. It does not add associations to the existing objects.
Import associations dialog

Processing

After pressing start, a monitor dialog is displayed, showing the progress. Upon completion, a summary dialog is shown. Note: processing can be run in the background, allowing you to continue your work. The progress view will display the result. Clicking the link will display the same summary as above (when running in the foreground).
Import associations dialog

Version History

Version 0.4, 04-Feb-2018

• Added support for Designer 4.6 LDAP and non-LDAP.

Version 0.3.1, 23-June-2015

• Bug fix: fixed issue with backslash during import of associations.

Version 0.3.0, 21-June-2015

• Bug fix: fixed NPE during import of associations.

Version 0.2.0, 5-March-2014

• Bug fix: fixed issues with modifying the state of not-associated objects.
• Enh.: remember last settings.

Version 0.1.0, 14-SEPT-2012

• First release (beta).

Know limitations

ldap paged search is somehow not fully implemented in eDirectory. Some ldap searches will return duplicate entries when using paged searches. This is a known eDirectory issue. In order to work around this somehow, the search is sometimes split up into multiple searches. Still, no guarantee is given that objects will not be synchronized twice.

Generic File Driver

• Introduction
• Version History
• Installation
• Policies
• Configuration
  • Driver
  • Subscriber
  • Common Subscriber Options
  • New File Strategy
  • File Name Strategies
  • File Write Strategies
  • Publisher
  • Common Publisher Options
  • RegExp File Locator
  • File Property Sorter
  • File Read Strategies
  • Logging Option
  • File Clean Strategies

Introduction

The Generic File Driver is similar to the Text Driver shipped with IDM, but has more options, and has the capability to read virtually any file type. Out of the box, the following file types are supported: XML, CSV and XLS (the latter using POI from Apache).

Main differentiators to NetIQ's driver:

• Produces and consumes regular NDS documents (no need for XSLT conversion).
• Accepts add, modify or instance documents on the subscriber channel.
• Auto-query back for missing attributes on modify commands.
• GCV (Ecmascript) for generating associations and source DN's.
• Publisher channels processes files in a streaming mode. A maximum of 20 records are prefetched/buffered in memory (whenever support by the file type: CSV and XML).
• Publisher channel supports interruption (driver stop command) during file processing.
• Publisher channel can add metadata about the file or record processed (eg: record number, filename, last record indicator, etc).

NetIQ community members' rating: (5 stars, 4 votes, average 5 out of 5)

This software is MPL 2.0

Version History

version 1.2, 11-DEC-2018 (base package 1.0, sample config package 0.1)

• Shim bug: Shim locked when an empty CSV file was published, and the driver was configured to use the schema from the CSV file (the shim waited for the schema forever).
• Shim feature: Converted status to heartbeat events (to have the driverstorage updated).
• Shim feature: Moving files now uses java 7 native move file method (and falls back to java 6 code when this fails). Note: shim is still compiled for older java 6 jre, but with java 7 source compatibility.

version 1.1, 17-JAN-2018 (package 0.9, sample config package 0.1)

• Shim feature: Added option to define sensitive attributes
• Shim feature: Added ImageFileReader and RawFileReader
• Shim feature: CSVFileWriter: added option to always surround fields with quotes.

version 1.0, 9-JAN-2017 (package 0.8, sample config package 0.1)

• Shim feature: provided an uncompressed jar, so that the shim can be discovered by the Windows remote loader. See forum post. Thanks AB!
• Release bug: Version 0.9 release seemed to have failed to include some features/bug fixes. All 0.9 fixes and features should now actually be included.

version 0.9, 7-DEC-2015 (package 0.8, sample config package 0.1)

• Shim bug: in certain conditions, doing a retry in the publisher channel on the first record of a file, caused the file not to be reprocessed when restarting the driver.
• Shim bug: forum issue with time limited subscriber files.
• Package feature: split up of package into base and sample configuration.
• Package feature: added Managed System Information Package as an optional package.
• Shim feature – publisher – new option to generate a log file for the publisher channel: every status message received as a result of a record, is written to a log file. Base package includes new configuration parameters.
• Shim feature – subscriber – added option to keep a local copy of the generated files.
• Shim feature – subscriber – changed file closing when maxRecordsPerFile is defined. Previously, when maxRecordsPerFile was set to n, the file was closed just before writing the (n+1)th file. New implementation closes the file after writing the nth record.

version 0.8A, 3-MARCH-2015 (package 0.7)

• Shim feature – general – reverted back to java 1.6 compatible ‘move file’ operation.

version 0.8, 14-SEPT-2014 (package 0.7)

• Package bug: fixed typo in example xslt template for XML publisher files. See forum thread.
• Technical – general refactoring of the shim.
• Shim feature - publisher - added option to remove old publisher files. Disabled by default for backward compatibility.
• Shim feature - publisher - added option to disable the publisher channel.
• Shim feature - publisher - added additional options in the command to generate ("Dynamic Input based" and "Delete")
• Shim feature - subscriber - added option to disable the "max number of records in generated files" by setting it to 0 (zero).
• Shim feature - subscriber - when a modify command is 'empty' (=all fields defined in the schema are missing), but the content indicates that the output file should be closed (eg: the file close field is set to true), no query back is performed in order to add a record to the output file. It is assumed that the only purpose of this modify command is to close the output file.
• Shim feature - general - added option to replace illegal ECMA script characters in field names (eg for calculating the association). Disabled by default for backward compatibility.
• Shim feature - general - added option to include the driver’s ECMA script libraries into the shim (for complex ECMA script operation). Disabled by default for backward compatibility.

version 0.7b, 14-MAY-2014 (package 0.6)

• Shim bug: case issue with the new configuration parameter to configure the command generated on the publisher channel

version 0.7, 07-MAY-2014 (package 0.6)

• Shim bug: specifying no encoding on the XML reader caused an Exception (see also this cool solution post)
• Shim feature: added option to configure the command generated on the publisher channel (add, modify or dynamic).
• Package bug: xml and xls file readers had invalid class names (see the same cool solutions post)
• Package feature: changed xls entry field to multi-line

version 0.6, 28-FEB-2014 (package 0.5)

• Shim/package feature: Added configuration parameter to use Quartz scheduler to close subscriber generated files using a cron-like expression string.
• Shim feature: Added slf4j binding (separate jar) for Idm tracing, allowing Quartz logs to be shown in idm trace files.
• Shim bug: possible NPE when the publisher folder becomes unreachable (eg due to network outage).
• Shim feature: Added option for every strategy to receive driver shutdown notification by implementing IShutdown.
• Shim bug/feature: Return an empty instance document when we receive an associated query on the subscriber channel. Reason: support of the merge attribute filter rules in case of a 'sync' event.

version 0.5, 21-JAN-2013 (package 0.4)

• Shim bug: Fixed an issue wrt the driver not adding the metadata on the publisher channel.

version 0.4, 24-APR-2012 (package 0.4)

• Shim bug: Fixed an issue with the driver schema (appeared to be hard coded).
• Shim feature: Added beta support for query back. Query back only supported on publisher channel (when reading in a file). Feedback welcome.
• Package bug: Fixed an issue that caused the driver configuration not to be imported when the package was imported.
• Package bug: Changed creation policy from validating the Given Name to validating the Surname.
• Package feature: Added package 'Delimited Text Driver Password Synchronization Package' as an optional package for this driver.
• Package feature: Added remote loader prompts to the package.

version 0.3, 12-OCT-2011

• Shim feature: automatic association and src-dn calculation (using ecmascript in the driver configuration). No longer need for style sheets for just creating association and src-dn values. The driver shim now does this for you.
• Shim bug: fixed missing remove-all-values on the publisher documents.
• Shim feature: added the ability (csv output files) to flush the file buffer after every record written (useful for when you are developing/testing your driver).
• Shim feature: when receiving a modify event on the subscriber channel, the shim will query back for all missing attribute values in order to be able to write a complete record (no need for a stylesheet to query for missing attributes and keeping this stylesheet in sync with the schema).
• Shim bug/feature: the subscriber channel now fully support add, modify and instance events. Instance and add events are assumed to contain all data and do not perform a query back. Only the modify event performs a query back

Initial release, back in 2007

The original version I wrote in 2007 is located at http://www.novell.com/coolsolutions/tools/18671.html. It did not support packages or any of the cool things we can do now, but it already had the extensible architecture and same basic principles (produce and consume NDS, add meta data,...).
See the referenced version for a list of the initial features.

Installation

I am in the process of migrating all my plugins to github, including an automated update site. This plugin is however not yet migrated (and since it also contains a shim will never be 100% available via an Eclipse/Designer update site). Download is available from Cool solutions.

The file attached to the Cool Solution (GenFileDriver_xxx.zip) contains (beside some documentation) all required jar files (the actual shim and 2 packages) and a lib folder. The lib folder contains some optional jars.

IDM Designer

Package: SVCGENFILEB_xxx.jar This is the base package. Import this package into your designer project. Drag and drop the "Delimited Text" driver that is located under "Tools" and select the "Generic File Base" package, answering all the questions as needed.

Package: SVCGENFILES_xxx.jar This is a sample configuration. Import this package into your designer project.

From this point on, you can use this driver and modify it as any other driver in Designer.

IDM Server

Shim: GenFileDriverShim_xxx.jar Copy the shim to you eDirectory server (DirXML's classes folder). If you want to use XLS support, also download poi from Apache. Restart eDirectory after this. This shim should run on any IDM system where you can run a remote loader or IDM engine.
Note: if you have an older version of the shim, be sure to delete that jar.

lib folder: This folder contains some optional jars. These are only required if you plan to use quartz scheduler to close the generated files using a cron-like string. Jars required when doing this are: quartz-2.2.1.jar, slf4j-api-1.7.5.jar and slf4j-idmTrace.jar.

Policies

This driver only has 6 policies; all of them for the publisher channel. 3 of the policies are purely for the publisher logfile generation, the 3 others are traditional publisher policies (matching, creation and placement). If required, you could add the password policies as available from the standard text driver.

SVCGENFILES-itp-InitPubLogMessage
Initialize the pubLogMessage variable for all add and modify operations. The variable pubLogMessage will be used for the publisher log file messages.

SVCGENFILES-pub-etp-StripInitParams
Strip the publisher state information from the document. If left in the document, it will generate an additional status message, resulting in an additional entry in the publisher log file. In order to prevent this additional log entry, we strip the publisher state information after it has been processed (around schema mapping).

SVCGENFILEB-pub-mp
Match on Internet Email Address if any available.

SVCGENFILEB-pub-cp
Checks for mandatory attributes:

• Surname: required due to eDir constraints
• Internet Email Address: required due to use in matching policy

SVCGENFILEB-pub-pp
Placement based on a driverset GCV (idv.dit.data.users).

SVCGENFILES-otp-PubLogMessage
Extend the status message to include the variable pubLogMessage if valued.

Configuration

All configuration options should have a good description. For completeness, the configuration is also described here:

• Driver
• Subscriber
• Common Subscriber Options
• New File Strategy
• File Name Strategies
• File Write Strategies
• Publisher
• Common Publisher Options
• RegExp File Locator
• File Property Sorter
• File Read Strategies
• Logging Option
• File Clean Strategies

Driver Configuration

Option	Description	Example value
Schema	Field Names (Field1,Field2,Field3)	LastName,FirstName,Title,Email, WorkPhone,Fax,WirelessP
objectClass	Object Class Name	User
association-field-script	ECMA script for generating the association associations (parameters are in the application scope). If non is provided, you are responsible for creating the association.	Email
source-field-script	ECMA script for generating the src-dn attribute (parameters are in the application scope). If non is provided, you are responsible for creating the association.	FirstName +' '+ LastName
js-processing	What should be done with invalid characters in the javascript field identifiers (eg: what if one of the fieldnames contains a space as in "Full Name"). NoProcessing: do nothing. You are responsible of making sure that all field names are valid java identifiers (including those not used in you java script). Replace: all illegal characters will be replaced be the given character	NoProcessing
js-replacement-char	In case of ‘Replace’: character to use to replace the illegal characters.	_
include-ecma-libraries	Should the shim include the ECMA script libraries linked to the driver for evaluating the various ECMA script. Set this to 'no' if the configuration does not depend on the ECMA script libraries (ECMA script configuration parameters above only contain simple, one-line scripts). Optimization parameter.	True

Subscriber Configuration

The subscriber configuration has the following 4 sections to configure:

• Common Subscriber Options
• New File Strategy
• File Name Strategies
  • GUID Filename Strategy
  • SimpleDateFormat Filename Strategy
  • Ecmascript Filename Strategy
• File Write Strategies
  • CSV File Writer
  • XML File Writer
  • XLS File Writer

Common Subscriber Options

These options are common for the subscriber, independent of the strategies used.

Option	Description	Example value
sub_FileStartStrategy	File Start Strategy (Class): Current implementations: info.vancauwenberge.filedriver.filestart.BasicNewFileDecider	info.vancauwenberge.filedriver.filestart.BasicNewFileDecider
sub_FileNameStrategy	File Name Strategy (Class): Current implementations: info.vancauwenberge.filedriver.filename.GUIDFileNameStrategy info.vancauwenberge.filedriver.filename.SimpleDateFormatFileNameStrategy info.vancauwenberge.filedriver.filename.EcmaScriptFileNameStrategy	info.vancauwenberge.filedriver.filename.SimpleDateFormatFileNameStrategy
sub_FileWriteStrategy	File Write Strategy (Class): Current implementations: info.vancauwenberge.filedriver.filewriter.CSVFileWriter info.vancauwenberge.filedriver.filewriter.XMLFileWriter info.vancauwenberge.filedriver.filewriter.XLSFileWriter	info.vancauwenberge.filedriver.filewriter.XMLFileWriter
sub_WorkDir	Work directory. All files will be created initially in the directory. When the files are closed, they are moved to the sub_DestFolder.	C:\workspace\DirXMLFileDriver\temp
sub_DestFolder	Destination directory:	C:\workspace\DirXMLFileDriver

Basic New File Decider

This strategy decides when a new file should be started. The BasicNewFileDecider has several options included: maximum number of records, maximum age of the file, idle time or a manual control based on a field in the Add event received.

Option	Description	Example value
newFile_MaxRecords	NewFile: Maximum number of data records in a file. Whenever this maximum is reached, a new file is started (unless overruled by newFile_FieldName). Set 0 to disable this.	100
newFile_MaxFileAge	NewFile: Maximum 'age' of a file (in seconds). The age counter starts from writing the first record to the file. As soon as the file is older then the given age, the next record will be written in a new file. Set 0 to disable this.	60
newFile_InactiveSaveInterval	NewFile: Save file after nnn seconds of inactivity. As soon as the driver does not receive new events whitin the given amount of seconds, the file is closed. A new file is started as soon as a new record needs to be written. Set 0 to disable this.	0
newFile_FieldName	NewFile: Fieldname used to manually control the creation of new files. If the value of this field is 'true', a new file will be started for adding the current record. Any other value will not start a new file. Note: as soon as this field is present, it overrules the maxRecord check. If you want both (manual and maxrecord), only set the field when it should be set to 'true'.
newFile_CronExpression	Cron string (Quartz) to schedule file closure. Leave empty to disable quartz scheduling. Note: If enabled, this requires the addition of other jar's from the lib folder into you eDirectory classes folder: quartz.jar, slf4j-api.jar and slf4j-idmTrace.jar.

File Name Strategies

File name strategies define how the name of the outputfile is defined. The following strategies are supported:

• GUID Filename Strategy
• SimpleDateFormat Filename Strategy
• Ecmascript Filename Strategy

GUIDFileNameStrategy

The GUIDFileNameStrategy generates filenames containing a unique GUID. You have the option to prefix the GUID and to postfix the GUID. The postfix is typically a file extentions. The new filename will be <prefix>GUID<postfix>

Option	Description	Example value
GUIDNamer_FilePostFix	GUID FileNamer: Optional postfix for the filename (eg: file extention).	.csv.out
GUIDNamer_FilePreFix	GUID FileNamer: Optional prefix for the filename.	NEW_FILE_

SimpleDateFormat File Name Strategy

The SimpleDateFormatFileNameStrategy generates filesnames containing a date/time. Internally, a SimpleDateFormat is used.

Option	Description	Example value
simpleDateNamer_FormatString	Date FileNamer: Date format string(according to java.util.SimpleDateFormat).	'NEW'_'FILE'_yyyyMMdd-HHmmssSSS.'out'
simpleDateNamer_TimeZone	GUID FileNamer: Timezone for the dataformatter. Leave blank to use system default timezone.

EcmaScript File Name Strategy

The EcmaScriptFileNameStrategy uses an ECMA script expression to generate the file name. All record fields are defined as contextual parameters in the ECMA script.

Eg: if your schema has a field ‘Email’, you can write the ECMA script Email+”.csv”, generating a file name like test@example.com.csv.

Option	Description	Example value
ecmaNamer_script.js	The ecma script	Email+”.csv”,

File Write Strategies

File writer strategies define the format of the file written. The following formats are supported out of the box:

• CSV File Writer
• XML File Writer
• XLS File Writer

CSV File Writer

The CSVFileWriter writes records in a CSV format. You can specify the seperator independent from the seperator from the CSV reader.

Option	Description	Example value
csvWriter_WriteHeader	CSVWriter: Write a header record(true/false):	true
csvWriter_Seperator	CSVWriter: Seperator character:	,
csvWriter_ForcedEncoding	CSVWriter: Forced file encoding. Leave blank to use system default encoding.:	UTF-8

XML File Writer

The XMLFileWriter writes the data in an XML file. The format is the same is what is expected in the XMLFileReader. Optionally a post xslt can be applied to get a different format.

Option	Description	Example value
xmlWriter_RootName	XMLWriter: Root element name.:	root
xmlWriter_RecordName	XMLWriter: Record element name:	record
xmlWriter_ForcedEncoding	XMLWriter: Forced file encoding. Leave blank to use system default encoding.:	UTF-8
xmlWriter_PostXSL	XMLWriter: Xsl to apply after saving the XML file (leave empty for no postprocessing):	Hover to see example

XLS File Writer

The XLSFileWriter writes all records into one sheet(tab) of an XLS file.

Note: this was not (re)tested recently. The original development and POI jar was way back around 2006. If you have any experience with the XLSFileWriter, please let me know.

Option	Description	Example value
xlsWriter_SheetName	XLSWriter: Sheet/tab name.	Sheet1
xlsWriter_AddHeader	XLSWriter: Add a header row (true/false)	true

Common Publisher Options

These options are for the publisher, independent of any strategy used.

Option	Description	Example value
pub_impl_class	Enable or disable publisher channel (class). Current implementations: info.vancauwenberge.filedriver.filepublisher.FileDriverPublicationShimImpl info.vancauwenberge.filedriver.filepublisher.DisabledPublicationShimImpl	Enabled
pub_sensitiveAttributes	Publisher: Comma separated list of sensitive attributes. These attribute values will have the "is-sensitive" attribute set when submitted on the publisher channel. You can use this to mask actual sensitive data or, in case of the ImageFileReader, mask the base64 image string that fills up the log file.	imageBytes
pub_fileLocator	Publisher: File Locator Strategy (Class): Current implementations: info.vancauwenberge.filedriver. filelocator.RegExpFileLocator	info.vancauwenberge.filedriver.filelocator.RegExpFileLocator
pub_FileSorter	Publisher: File Sorter Strategy (Class). Leave empty for no sorting: Current implementations: info.vancauwenberge.filedriver.filesorter.FilePropertySorter	info.vancauwenberge.filedriver.filesorter.FilePropertySorter
pub_fileReader	Publisher: File Read Strategy (Class): Current implementations: info.vancauwenberge.filedriver.filereader.csv.CSVFileReader info.vancauwenberge.filedriver.filereader.xml.XMLFileReader info.vancauwenberge.filedriver.filereader.xls.XLSFileReader info.vancauwenberge.filedriver.filereader.image.ImageFileReader info.vancauwenberge.filedriver.filereader.raw.RawFileReader	info.vancauwenberge.filedriver.filereader.csv.CSVFileReader
pub_pollingInterval	Publisher: Polling interval (seconds):	80
pub_metaData	Publisher: List of meta data elements that should be added(recordNumber,isLastRecord,filePath,fileName,fileSize) . The schema as given in the driver parameter 'schema' will be extended with all meta data elements listed.	recordNumber,isLastRecord
pub_heartbeatInterval	Publisher: Heartbeat interval (minutes):	1
pub_workDir	Publisher: Temporary work folder. All files are moved to a subdirectory (containing a date/time) of this directory prior to reading the file.	C:\workspace\DirXMLFileDriver\temp
pub_command	Generated command: add (default), modify, delete, ‘dynamic’ or ‘Dynamic Input based’. Add, modify and delete are static options (every event is the same). In Dynamic mode, the driver will first generate a query based on association. If One instance is returned, it will generate a modify, otherwise it will generate an add. ‘Dynamic Input based’ will first test to see if it should generate a delete event. If not, if follows the ‘Dynamic mode’. This gives in certain scenarios more freedom to use the filter merge authority setting (by avoiding a merge on the publisher channel). Note: modify events will generate a query-back when a merge is needed. This can have serious performance consequences! For this reason, using 'modify' is not advised.	Add (Default)
pub_command_fieldName	In case of ‘Dynamic Input based’, the field that determines if this is a delete event.	operation
pub_command_delete_regexp	In case of ‘Dynamic Input based’, contains the regexp that identifies delete entries.	D
pub_fileClean	How should processed files be cleaned up. Current implementations: info.vancauwenberge.filedriver.filepubclean.NoPubCleaningStrategy info.vancauwenberge.filedriver.filepubclean.AgePubCleaningStrategy
pub_IsLogging	Enable or disable publisher file process logging	Enabled

File Read Strategies

File readers define the format of the file that can be read. Currently supported formats are:

• CSV File Reader
• XML File Reader
• XLS File Reader
• Image File Reader
• Raw File Reader

CSV File Reader

The CSVFileReader reads CSV files (with or without header). It has the option to skip empty lines, use a given seperator, use the header names (if any) instead of the given driver schema as field names, etc...

Option	Description	Example value
csvReader_skipEmptyLines	CSVReader: skip empty lines (true/false)	True
csvReader_UseHeaderNames	CSVReader: use the header names (true) or the driver schema given (false). Using the header names has the advantage that the order of the columns can change, without affecting your code. The downside is that if the header names changes (eg: change in case), your code will no longer work.	True
csvReader_hasHeader	CSVReader: has the CSV file a header (true/false)	True
csvReader_forcedEncoding	CSVReader: forced enoding of the xml file. Leave blank to use system default encoding.	UTF-8
csvReader_seperator	CSVReader: field separator	,

XML File Reader

The XMLFileReader reads XML files. The expected format is ‘flat’:

<root>
  <aRecord>
    <aField>avalue</aField>
    <anoherField>anotherValue</anotherField>
  </aRecord>
  <aRecord>
  ...
  </aRecord>
</root>

If the file is not in the given format, an optional preXslt can be applied to make it in this format. As with the CSV reader, you have the option to use the tag names (eg: aField in the example above) as field names, or use the schema given in the driver configuration.

Option	Description	Example value
xmlReader_useTagNames	XMLFileReader: Use the tag names from the XML document (true) or use the driver schema given (false). Using the tag names has the advantage that the order of the elements can change, without affecting your code. The downside is that if the element names changes (eg: change in case), your code will no longer work. The latter should not occur in an XML file, because XML is case sensitive (making at in fact another field when the case of the tag changes).	true
xmlReader_preXslt	XMLFileReader: xsl to apply prior to processing the XML file (leave empty for no pre-processing):	Hover to see example
xmlReader_forcedEncoding	XMLFileReader: forced enoding of the xml file. Leave blank to use the encoding attribute inside the xml file (or system default when attribute not given).

XLS File Reader

The XLS file reader reads 1 tab of an Excel sheet. This reader will read the complete XLS file in memory.

Note: this was not (re)tested recently. The original development and POI jar was way back around 2006. If you have any experience with the XLSFileWriter, please let me know.

Option	Description	Example value
xlsReader_SheetName	XLSReader: Name of the sheet within the XLS file to read.	Sheet1
xlsReader_HasHeader	XLSReader: has the XLS sheet a header (true/false)	true

Image File Reader

The Image file reader reads image files as supported by your java environment. By default, java supports PNG,GIF,JPEG and BMP. Since ImageReaders and Writers are java service (via SPI), this can be extended.
The ImageFileReader generates the following predefined fields:

• Properties of the image submitted on the pub. channel:
  • imageBytes: base64 encoded image bytes
  • imageHeight
  • imageWidth
  • imageFormat
• Properties of the original image:
  • srcHeight
  • srcWidth
  • srcFormat

On top of these, you can still configure the publisher to submit file name, path, etc as required. Note: Images will be read and resized/transcoded in memory. Be sure to have enough memory for the images. Only one image will be in memory, but this image will be duplicated (if resizing) or even tripled (if transcoding)

Option	Description	Example value
imgReader_resize	Resize the image (true/false)	True
imgReader_resizeMode	Mode of resizing. Exact: will resize to the given width and height, regardless of the input image. Aspect ratios might not be respected. Max dimensions: will resize keeping aspect ratios. When both height and width are >0, the image will be resized until either the width or the height are reached. When either height or width are zero, the image is resized to the given height or width, alculating the other for keeping aspect rations.	Exact
imgReader_resizePadding	Only when resize mode is Max dimensions. When padding is enabled, and both height and width >0, the image will be resized (keeping aspect ratio), centered, and the remaining spaces filled with the padding color.	No Padding
imgReader_resizePaddingColor	Only when Padding. Padding color is defiend as HEX RGB (eg:121212 for grey)	000000
imgReader_resizeY	New height of the image (in pixels). When resize mode is "Max dimensions": set to '0' to scale the width of image to "New width" and calculate the height to keep aspect ratio. At least "New height" or "New width" must be set > 0.	250
imgReader_resizeX	New width of the image (in pixels). When resize mode is "Max dimensions": set to '0' to scale the height of image to "New height" and calculate the width to keep aspect ratio. At least "New height" or "New width" must be set > 0.	200
imgReader_transcode	Transcode the image. (true/false) Caution: transcoding is not supported between all formats. Supports depends on source and destination type, but also on things like color map etc. 'png' seems to be working for most input types. transocding does not keep image meta data (author, shooting date, etc) Use at your own responsibility.	true
imgReader_transcodeFormat	Destination format. examples: png, gif, bmp, jpeg.	png

Raw File Reader

The Raw file reader reads arbitrary files, both binary and text files. The content is submitted as one attribute value.
Files will be read (and base64 encoded) in memory. Be sure to have enough memory depending on your expected file size. It is possible to limit the size of the files.
The RawFileReader generates the following predefined fields:

• rawData

On top of these, you can still configure the publisher to submit file name, path, etc as required.

Option	Description	Example value
rawReader_maxSize	Maximum file size (in bytes) to import. Make sure you have enough memory when you want to read large files. Set to '0' to disable this protection.	20000
rawReader_type	Binary or text. Binary files will be base64 encoded. Text files will be read (using the given encoding) and submitted as a string.	Binary
rawReader_typeText_Encoding	In case of text files: File encoding. Leave empty for platform default.	UTF-8

Publisher Configuration

The publisher configuration has the following options and strategies:

• Common Publisher Options
• RegExp File Locator
• File Property Sorter
• File Read Strategies
  • CSV File Reader
  • XML File Reader
  • XLS File Reader
  • Image File Reader
  • Raw File Reader
• Logging option
  • Common Option
  • CSV Logger
• File Clean Strategies
  • No Cleaning Strategy
  • Age Cleaning Strategy

RegExp File Locator Options

The RegExpFileLocator is responsible for finding files that need to be processed. The given implementation scans a source folder, and selects all files that match the given regular expression.

Option	Description	Example value
regExp-sourceFolder	RegExpFileLocator: Source folder for files:	C:\source
regExp-regExp	RegExpFileLocator: Regular expression for finding files.Use (?i) at start to make case ignore regexp:	(?i).*\.csv

FileProperty Sorter Options

The FilePropertySorter sorts the list of files returned by the file locator implementation, and sorts them (ascending or descending) on a given file property. The property can be any file attribute as exposed by the java file object.

Option	Description	Example value
fileSort_SortMethod	FileSorter: File sort methode:	getName
fileSort_SortOrderAsc	FileSorter: Sort ascending (true/false):	true

File Cleaners

The publisher cleaning strategy determines how the files received are actually removed from the file system. Following strategies are available:

• File Clean Strategies:
  • No Cleaning Strategy
  • Age Cleaning Strategy

No Cleaning Strategy

This publisher cleaning strategy does not do any cleaning. All published files are kept on the system. This strategy has no parameters.

Age Cleaning Strategy

This publisher cleaning strategy removes files that are older (last modify date) than a certain age.

Option	Description	Example value
pubclean_age_number	How many units should it be before cleaning the file	5
pubclean_age_unit	The units of the above number. Possible values: Minutes, Hours, Days, Weeks	Days
pubclean_age_test_interval_number	How frequent should the age of files be tested.	10
pubclean_age_test_interval_unit	The units of the above number. Possible values: Minutes, Hours, Days, Weeks	Minutes

Logging

The publisher logging allows generating a log file for each file received. Log records contain the status received as a response to the add or modify XDS commands submitted by the shim.

• Logging options:
  • Common Logging Options
  • CSV Logger Options

Common logging options

These are common logging settings for all publisher loggers (only one for now). Logging on the publisher channel will generate a log record for every status message received as a result of an add or modify event generated on the publisher channel.

Option	Description	Example value
pub_logFieldRecordNumber	Field name for the record number in the log file. Leave empty to not add it.	RecordNo
pub_logFieldStatus	Field name for the status (Error, Retry, Succes etc) in the log file. Leave empty to not add it.	Status
pub_logFieldStatusMessage	Field name for the status message in the log file. Leave empty to not add it.	StatusMessage
pub_logFieldEventID	Field name for the status event id in the log file. Leave empty to not add it.	StatusEventID
pub_FileLogger	File Logging Strategy (object class implementing IPublisherLoggerStrategy). Determining how the publisher generates logfiles of the files processed. Current implementations: info.vancauwenberge.filedriver.filelogger.PublisherCSVLogger: Log to a CSV file	info.vancauwenberge.filedriver.filelogger.PublisherCSVLogger

CSV Logger options

The Publisher CSVLogger will write the publisher logger to a CSV formatted file.

Option	Description	Example value
csvLogger_WriteHeader	csvLogger: Write a header record(true/false).	true
csvLogger_Seperator	csvLogger: Seperator character.	,
csvLogger_ForcedEncoding	csvLogger: Forced file encoding. Leave blank to use system default encoding.	UTF-8

Package Unlocker

• Intro
• Installation
• Version history

Introduction

When you are working with Designer and you create new packages, you can only modify the package in the project that created the package. Luckily, Designer allows you to use subversion, so you can share the project, resulting in all connected designer instances to share the same project, and thus being able to 'own' the same packages.

But what if you do not have subversion, or you are at a client that does not allow access to your subversion repository? In this case, you can import the custom packages, but you are unable to modify the content of the package or create a new version of this package. You can only copy a specific version to a new package, breaking your package structure.

This Designer extension allows you to mark a package (any package!) as 'Local', thus allowing you to create a new version of your package or add new content to the package.

Note: only use this on packages that you are actually owning. If not, the vendor of the package might release a conflicting version (and then you'll have a mess to clean up).

NetIQ community members' rating: (5 stars, 3 votes, average 5 out of 5)

This software is MPL 2.0

Installation

I am in the process of migrating all my plugins to github, including an automated update site. This plugin is however not yet migrated. Download is available from Cool solutions.

Note: this plugin should run on any system Designer runs on

Steps:

1. Change the extentions of the included file to '.jar' (Cool tools does not allow files with a jar extention) and copy the file to Designers plugin folder.
2. (Re)start designer, making sure your project is in 'developer' mode.
3. If you right-click on a locked package, you should have an additional menu item: ”Mark as 'Local'”.
4. Selecting this menu item, will request a confirmation of your action. Pressing 'OK' will unlock the package, allowing you to create a new version.

Note: this plugin also adds a log viewer to you packages: right click on any package and select “View changelog”. This is a log of the package as defined by the actions created by Novell.

Version History

Version 1.0.1, 16-Apr-2016

• Changed the warning text in agreement with NetIQ.
• Added about.html
• General cleanup of plugin definition
• Changed installation: jar instead of folder

Version 1.0.0, 05-Oct-2011

• Initial release, contains the package unlocker and a package changelog viewer