Demystifying file read as attachment in SOA 12c

 In cases when we just want to move a file(without transformation) from one SFTP/FTP/file location to the other SFTP/FTP/file location, we can read a file with or without schema and then perform the write operation to the target location. If we provide an exact file schema for conversion to XML, it takes a lot of heap memory to transform from a csv or ffd to an XML document. In cases when files are not simple text files or schema is not available, one way to do so is to read the file as an opaque object and write it to a target location. In doing so, we still use a heap memory of the WebLogic server. If we are dealing with very large files, this may impact server performance as the conversion to opaque elements will take place behind the scenes. Using opaque schema is never recommended to process huge files. One better way of handling such a scenario is to read a file as an attachment in SOA 12c. Generally, we have below question:

What is reading a file as an attachment means?

In this case, we are not loading the contents of the file in the memory of the Weblogic server i.e in BPEL flow we will not see the contents of the file loaded instead, a href to the actual file is seen. In this case, the Weblogic server leverages SOA-INFRA tables to load file data in the table and in return gives href to the flow. This href actually links to data stored in the SOA-INFRA table and can be used to write files to the different locations (in the same or different FTP servers). 

To understand better, let's take an example of flow which polls the FTP directory every 10 minutes and read as an attachment. Then, how we can find the payload from SOA-INFRA tables. 


Below are the steps to configure the file/FTP adapter in BPEL(SOA 12c).

Configuration Steps:

1. Create a SOA project in Jdevloper 12c.
2. Drag and drop FTP adapter from component pallet. Enter the service name. Click Next.
3. Select Define from operation and schema (specified later). Click Next.
4. Specify the source FTP JNDI name as configured in the Weblogic server. Click next.
5. Select operation type as Get File. Check the Read File As Attachment. If we know the character set, encoding, and Content-Type of the file, then specify else leave it blank. Click Next.
6. Specify the input directory from where we want to pick the file from. Check Archive processed files, and specify a directory for archive files if we want to archive the data after processing. Check Delete files after successful retrieval if we want to delete the file after processing. Click Next.
7. Provide the file name pattern or regex file name pattern and click Next.
8. Specify the polling frequency and click Next. Click on Finish to complete the adapter configuration.
9. Add a BPEL component with no service, i.e. define service later.
10. Join the wire from the FTP adapter to the BPEL component.

11. Open the BPEL component, and configure receive activity as shown below.
12. Click OK. Save and deploy the code on the Weblogic server.

Analyzing the read file as an attachment

Let's test the service and see the behaviour of file being read as an attachment.

1. Place the file in the FTP location and wait for it to get it picked.
2. Once the file is picked, log in to the em console. We will see an instance created.
3. Open the flow, here we will see only the href.
Now after seeing this, we might think about where did our data go. As we see only href. But where is our actual data?

Our data is safe within soa-infra database.

4. To query the file data and file attributes, connect to the SOA-INFRA database of the SOA instance.
5. Run the below query
select * from attachment where dbkey='href-value';
File data is stored in blob format within the attachment table.

References:


Please share your valuable feedback 😊😊😊


-

Comments

  1. Vibhav Singh28 November, 2021 10:09

    Good one

    ReplyDelete
  2. Naveen09 June, 2022 10:46

    Hi Pranav, can you please guide me on how read file content from href/db and send the same as an attachment in email notification. Thank you in advance

    ReplyDelete

Post a Comment

Popular posts from this blog

DateTime formatting using xp20:format-dateTime ()

xs:dateTime Format In XML, we generally use xs:dateTime(xs:xmlns="http://www.w3.org/2001/XMLSchema") data type to store date time data. The generic representation of date time, that is supported by xs:dateTime in xml: YYYY-MM-DDThh:mm:ss[.SSS][Z|(+|-)hh:mm] Example :  2019-09-11T10:16:31.943+05:30 Code Description Example YYYY represents a 4-digit year 2019 MM represents a 2-digit month 09 DD represents 2-digit date 11 hh represents 2-digit hours 10 mm represents minutes in 2-digits 16 ss represents seconds in 2-digits 31 SSS represents milliseconds 943 T represents time separator T Z represents UTC time zone or 0 time zone offset (+|-)hh:mm represents time zone offset +5:30 Few more valid examples : 2019-09-11T21:32:52, 2019-09-11T21:32:52+02:...
Read more

Create Delimited String from XML Nodes and Vice Versa in SOA 12c

A delimited string is a string representation of data separated by a delimiter(e.g. ","). A simple representation of the delimited string will look like. Delimited String Examples Sample-1-- Delimiter as ","        Value1,Value2,Value3   Sample-2-- Delimiter as "|"        Value1|Value2|Value3 If we want to represent the above set of delimited values in the form of an XML document under some root and parent tag. The representation will look like this. XML Example <root xmlns="http://test.com/sample/xml">   <Values>     <value>Value1</value>     <value>Value2</value>     <value>Value3</value>   </Values> </root> Sometimes we may need to convert values coming in delimited string to an xml document or vice versa. Since delimited strin...
Read more

Import and Export MDS artifacts in SOA 12c

Image
Oracle SOA-MDS (Metadata Store) repository can be used to access artifacts that can be shared among various SOA composites. For example, we have one common schema for a business fault that is consumed by various applications. Instead of having a localized copy of that schema file in each SOA composite, we can have it in a centralized repository (SOA-MDS). Another benefit of using SOA-MDS is that we can change the file at runtime and need not redeploy the complete code. While developing SOA composites, we use SOA design-time repository, later these changes can be synced to the SOA servers. In this article, we will discuss 2 ways to sync the artifacts available in the SOA design-time repository and the SOA servers. Establishing the connection to SOA-MDS in Jdeveloper. Using SOA em console or Fusion Middleware control console. Moving a file between SOA composite to SOA design-time MDS repository Let's assume, we created a businessFault.xsd which wi...
Read more