FLAM® Issue Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000888FL53. Subsystem, Driver, Plugin, SPIpublic2017-08-29 10:532017-10-23 12:53
ReporterFalk Reichbott 
Assigned ToUlrich Schwab 
PriorityhighSeverityfeatureReproducibilityN/A
StatusassignedResolutionopen 
PlatformGeneralOSGeneralOS VersionGeneral
Product Version5.1.15 
Target Version5.1.17Fixed in Version 
Summary0000888: Support an application transparent file conversion, compression and encryption solution like z/OS-Subsys for open systems
DescriptionCurrently FLAM supports subsystem functionality only on mainframe systems. To use FLAM as infrastructure it would be useful to support such a application transparent solution also on other platforms (UNIX, Windows).

Per logical file it must be possible to define read- und write-string for the physical representation of this file. This repräsentation can be a converted, compressed, encrypted local or remote version of the logical file.

For example: An application (SAP) writes a file for credit card ordering which must be encrypted. A system programmer makes a definition in a hidden file, that this logical file must be represented as an OpenPGP file in phsical form and the FLUC I/O subsystems make this conversion without any change to the application.
Additional InformationFUSE (Filesystem in Userspace) could be a soultion for UNIX systems
TagsNo tags attached.
Attached Files

- Relationships
related to 0000891assignedFalk Reichbott Make record and byte interface thread save 

-  Notes
(0001114)
Falk Reichbott (administrator)
2017-08-29 11:04

FUSE on AIX

https://www.ibm.com/developerworks/aix/library/au-spunix_newfs/index.html [^]
(0001116)
Falk Reichbott (administrator)
2017-08-31 09:23
edited on: 2017-08-31 12:23

Solution with FUSE

We implement a main (executable) for mount 'flucfuse' with standard fuse parameter (only write or read and write, only user or root or other, in forground or background, ...) and one optional file name with the write definitions in the CLP syntax below:

fluc.write(file='matchstring*with*wildcards'
   substitute[(format="format.string" file="write/output.string" [readable])]

where substitute is an array for multiple writes (we use n times fcbopen() and do n times fcbpwrite() for each handle) and only one of them can have the readable flag set. If no central config file name is given, a hidden file (.flucfuse.write) is used in each directory. New write definitions might be added on the fly. If the readable flag is defined, then the inverse format (must be implemented) and read (must be changed from pfIvr to pdIvr) strings are written to a hidden file (.flucfuse.read) in each directory with the CLP syntax below:

fluc.read(file='logicalfilename'
   substitute(format="format.string" file="read/input.string")
   attributes(filesize=4711))

This file is used for read requests in inverse form. The filesize for stat and other attributes are also stored in this directory extension.

Additionally we must finalise the chmod support, so that the user can define the access rights for the physical files in the "write/output.string".

The FLUC byte interface (FLCBYT) must be extended by the IVR functions to get the inverse strings for the read operation.

If multiple FIOs are defined in the "output.string" then only the first FIO definition must be used to build the inverse command, the other inverse strings must be comments, so that this implicit rule makes the read unique.

The byte interface gets 3 new function. First fcbpread() and fcbpwrite() which manage the offset on the FCB handle. The third will be an extension to the close function which provides the attribute string for the logical file. At each close of a converted file, the inverse entry is made if the readable flag is set and the read specification file is written.

Additionally all other FCB functions get a thread save version with errno, trcsiz and trcbuf as the first 3 parameter.

By linking the flucfuse main programm the LMAP must be checked to verfiy
that no static data area is present, in the implementation of the FLUC file system.

If a file don't fit with the match string a passthrough will be done. If a file matches, then the locical name is given in the state string to fcbopen() and will be converted to the pysiscal name based on the defined mapping rules in the "write/output.string". The physical name is given back in the inverse read string.

To automate reload of a central config file inotify should be used.
Changes to config files within the work tree are seen anyway and can be used
instantly.

The implementation is limited to sequential read and write. If an application does seek and makes updates, an I/O error is flagged.

A special case is a seek_end, ftell and rewind. This cases are covered by a complete read (if possible) and a reopen of the file afterwards.

There will be another filesystem called flucfusecheck which makes passthrough, and logs per file all functions and check the offsets for sequential read and write. This file system can be mounted to figure out, which files of this mountpoint can be converted by a write specification and which files will fail.

Questions:

Is a central config file for write operations useful?
The implementation would be more simple if the configuration is done as an extension to each directory only based on the file name, without any path. For example you mount the flucfuse on directory pgpfile in your home directory and you make the definition in the file pgpfile/.flucfuse.write.

fluc.write(file='*'
   substitute(format="format.bin()" file="write.bin(encr.pgp(user='~'))"
              readable)

In this case each file of this folder will be OpenPGP encrypted under the own user id.

Another solution could be that the config file at mount defines the conversion for the whole mountpoint. In this case there is no passthrough implemented. All files under this mount point are converted as defined at mount time. This requires a mount per write definition. Would this be more useful?

(0001117)
Falk Reichbott (administrator)
2017-08-31 10:23
edited on: 2017-08-31 12:26

For the read specification it makes more sense to work with a hidden directory where for each converted file a hidden file is stored with the inverse read specification.

/home/user/OpenPGPFiles/.flucfuse.write
/home/user/OpenPGPFiles/.flucfuse.read/.myfirstfile.txt.flucfuse.read
/home/user/OpenPGPFiles/.flucfuse.read/.mysecondfile.txt.flucfuse.read
/home/user/OpenPGPFiles/myfirstfile.txt.pgp
/home/user/OpenPGPFiles/mysecondfile.txt.pgp

This makes the access to the read specification easier.

(0001119)
Falk Reichbott (administrator)
2017-08-31 16:56
edited on: 2017-09-01 10:01

FUSE on AIX is not possible based on missing kernel support for it.

IBM recommend to use NFS on AIX in conjuction with a LINUX server where FUSE exists.

(0001120)
Falk Reichbott (administrator)
2017-08-31 17:06

A reposition to a previos location at read, can be simply implemented, see below:

if (uiOffset<psHdl->uiOffset) {
   psHdl=fcbreopen(psHdl);//makes close and open
   fcbpos(psHdl,uiOffset);
} else if (uiOffset>psHdl->uiOffset) {
   fcbpos(psHdl,uiOffset);
}
fcbread(psHdl,uiLen,pcDat);

The function fcbreopen and fcbpos must be implemented.
(0001121)
Falk Reichbott (administrator)
2017-09-01 13:02
edited on: 2017-09-08 09:33

An additional hidden file containing the last read or write error must be added to the hidden folder described above.

/home/user/OpenPGPFiles/.flucfuse.read/.myfirstfile.txt.flucfuse.read.errortrace
/home/user/OpenPGPFiles/.flucfuse.read/.myfirstfile.txt.flucfuse.write.errortrace


- Issue History
Date Modified Username Field Change
2017-08-29 10:53 Falk Reichbott New Issue
2017-08-29 10:53 Falk Reichbott Status new => assigned
2017-08-29 10:53 Falk Reichbott Assigned To => Ulrich Schwab
2017-08-29 11:04 Falk Reichbott Note Added: 0001114
2017-08-31 09:23 Falk Reichbott Note Added: 0001116
2017-08-31 10:23 Falk Reichbott Note Added: 0001117
2017-08-31 12:23 Ulrich Schwab Note Edited: 0001116 View Revisions
2017-08-31 12:26 Ulrich Schwab Note Edited: 0001117 View Revisions
2017-08-31 16:56 Falk Reichbott Note Added: 0001119
2017-08-31 17:06 Falk Reichbott Note Added: 0001120
2017-09-01 10:01 Ulrich Schwab Note Edited: 0001119 View Revisions
2017-09-01 13:02 Falk Reichbott Note Added: 0001121
2017-09-07 14:31 Falk Reichbott Description Updated View Revisions
2017-09-08 09:33 Ulrich Schwab Note Edited: 0001121 View Revisions
2017-10-23 12:53 Falk Reichbott Relationship added related to 0000891


Copyright © 2000 - 2017 MantisBT Team
Powered by Mantis Bugtracker