JSON on DecipherMiddleware

Playing With JSON data in Oracle Database- Part1

Tue, 05 Dec 2023 19:43:21 +0000

Whenever we hear Oracle Database, we imagine storing data in rows and columns. The data stored in cells generally is in primitive data types(char, varchar2, int, date, etc.). What if we need to store JSON data in the table. The first thing that may come into our minds is to convert JSON data to string(varchar2) or have data as clob/blob and insert it into the table. But with this if we need to query a particular field or update it, tricky one. This blog is a part of a series. The scope includes, creation of table having column with JSON datatype, insertion of JSON data and querying the JSON data.

Umm.. How to do? 🔗

Let’s put the developer hat on! and start exploring. In this blog, Oracle Database 23c free is used.

Create a table with a column to store JSON data 🔗

JSON data can be used natively with RDBMS features in Oracle Database 23c. Let’s start the journey by creating a simple database table for storing JSON data.

Create a sequence SEQ_PO for the id column.

create sequence SEQ_PO start with 1000 increment by 1 NOCACHE NOMAXVALUE;

Create a table q_purchaseorder to store purchase order documents in JSON format.

CREATE TABLE q_purchaseorder
(id NUMBER DEFAULT ON NULL SEQ_PO.NEXTVAL PRIMARY KEY,
date_loaded TIMESTAMP (6) WITH TIME ZONE,
 po_document JSON);

If, we want to store the JSON data in Varchar2, then we need to add a constraint “ensure_json” to check if the data is in JSON format. For example:

CREATE TABLE q_purchaseorder
(id NUMBER DEFAULT ON NULL SEQ_PO.NEXTVAL PRIMARY KEY,
date_loaded TIMESTAMP (6) WITH TIME ZONE,
po_document VARCHAR2 (23767)
CONSTRAINT ensure_json CHECK (po_document is json));

The table q_purchaseorder is created.

Insert data to the JSON column 🔗

Let’s insert some purchase orders JSON data to the newly created table in po_document column.

INSERT INTO q_purchaseorder (date_loaded,po_document) VALUES (
 SYSTIMESTAMP, '{
 "pONumber": "1001",
 "reference": "dm-1001",
 "requestor": "One More Query",
 "user": "dm_1",
 "costCenter": "CC1",
 "shippingDetails": {
 "name": "Test 1",
 "address": {
 "street": "Street 1",
 "city": "Amritsar",
 "state": "Punjab",
 "zipCode": "143001",
 "country": "India"
 },
 "phone": "+91-99999-99999"
 },
 "message": "Courier",
 "lineItems": [
 {
 "itemNumber": 1,
 "part": {
 "description": "Test product1",
 "unitPrice": 200,
 "uPCCode": "123456789"
 },
 "quantity": 8.0
 },
 {
 "itemNumber": 2,
 "part": {
 "description": "Test product2",
 "unitPrice": 250,
 "uPCCode": "987654321"
 },
 "quantity": 5
 }
 ]
}'
);

The above JSON record gets inserted into the table.

select * from q_purchaseorder;

Query the JSON data 🔗

The easiest way to query the JSON data is to use dot notations. The JSON data in the column must be well formed. If the data type is not JSON, varchar2, clob/blob column must have “is json “ constraint. Taking the data inserted in the previous step as an example, below are the various queries to extract values from different JSON fields.

Query the PO number. 🔗

SELECT po.PO_DOCUMENT.pONumber FROM q_purchaseorder po;

Query the unit price for second line item. 🔗

SELECT po.PO_DOCUMENT.lineItems[1].part.unitPrice FROM q_purchaseorder po;

The above two examples output the data as JSON object instead of SQL number or SQL STRING, as the column datatype is JSON. If datatype of the column is not JSON, the query will give a result in VARCHAR2 format. An item method, such as number(), string(), etc., can convert JSON/VARCHAR2 output from the query into the desired format. For example:

Query the quantity for the second line item and output in number format. 🔗

SELECT po.PO_DOCUMENT.lineItems[0].quantity.number() FROM q_purchaseorder po where po.id=1000;;

Another way to query data is to use JSON_VALUE SQL function. For example:

Query the PO Number in number format using JSON_VALUE SQL function. 🔗

SELECT JSON_VALUE(po_document, '$.pONumber.number()') from q_purchaseorder where po.id=1000;;

Pretty simple, yeah? Let’s make it a bit more complex.

How to fetch an item description from purchase order JSON where unit price is less than 225?

Scratching the head!!!! Let’s use json_query sql function along with filter conditions to accomplish the task.

SELECT json_query(po.po_document, '$.lineItems[*].part?(@.unitPrice < 225).description' with ARRAY wrapper)
FROM q_purchaseorder po
WHERE po.id=1000;

WITH ARRAY WRAPPER is used to wrap the results in array, as output may contain more than one values.

References: 🔗

Please feel free to share your valuable feedback ! 😊😊😊

Handling XML response with or without namespace in Postman

Wed, 10 Mar 2021 16:34:00 +0000

While testing an XML response in postman, we need to convert XML data to JSON in order to traverse child nodes of an XML document. XML documents can be with/without namespaces. If XML documents have namespaces, then those are declared as attributes to the XML tag. There may or may not be prefixes associated with XML tags containing namespaces. There are different methods or ways to convert XML to JSON, a few of which are described in this blog.

XML to JSON without namespaces -- using xml2Json
XML to JSON with namespaces -- using xml2js

Postman version: 8.0.7
Libraries : xml2Json, xml2js

XML to JSON without namespaces 🔗

Let’s assume we are getting the below XML response from Webservice.

<ROOT>
 <Account>
 <name>Name1</name>
 <id>123123</id>
 <address>
 <street>Street1</street>
 <city>City1</city>
 <state>State1</state>
 <Country>Country1</Country>
 </address>
 </Account>
 <Account>
 <name>Name2</name>
 <id>124234</id>
 <address>
 <street>Street2</street>
 <city>City</city>
 <state>State1</state>
 <Country>Country1</Country>
 </address>
 </Account>
</ROOT>

To convert this XML payload without namespaces, we can use the below code snippet in Postman.

Code: 🔗

const responseJson = xml2Json(pm.response.text());//fetch xml data in text/string
console.log(JSON.stringify(responseJson));//logging to postman console

The JSON converted data will be logged in the Postman console as below:

Pretty print:

If we want to log the value of city under the address element of second Account.

const responseJson = xml2Json(pm.response.text());//fetch xml data in text
const city=responseJson.ROOT.Account[1].address.city;//fetch the desired city value
console.log(city);//logging to postman console

Postman console will log the value as “City2 “.

In the above example, XML tag does not contain any namespaces. What if XML elements have namespaces and have different namespaces at each level of hierarchy. The next section gives various tips and tricks that can be used to tackle such scenarios.

XML to JSON with namespaces 🔗

Let’s take another example of XML response which contains namespaces and prefixes.

<tns:ROOT tns:xmlns="http://samplenamespace.com/v1/target">
 <tns:Account acc:xmlns="http://samplenamespace.com/v1/target/account">
 <acc:name>Name1</acc:name>
 <acc:id>123123</acc:id>
 <acc:address add:xmlns="http://samplenamespace.com/v1/target/account/address">
 <add:street>Street1</add:street>
 <add:city>City1</add:city>
 <add:state>State1</add:state>
 <add:Country>Country1</add:Country>
 </acc:address>
 </tns:Account>
 <tns:Account acc:xmlns="http://samplenamespace.com/v1/target/account">
 <acc:name>Name2</acc:name>
 <acc:id>124234</acc:id>
 <acc:address add:xmlns="http://samplenamespace.com/v1/target/account/address">
 <add:street>Street2</add:street>
 <add:city>City2</add:city>
 <add:state>State2</add:state>
 <add:Country>Country2</add:Country>
 </acc:address>
 </tns:Account>
</tns:ROOT>

If we use the same snippet as used in previous example i.e.

const responseJson = xml2Json(pm.response.text());//fetch xml data in text/string
console.log(JSON.stringify(responseJson));//logging to postman console

Then output from console will look like.
Pretty print

In this case, logging the value of **city **under the **address **element of the second Account becomes difficult as now the prefix is added and the key has “:” character which results in an invalid Js/JSON reference as below.

1 One way to reach the particular value is: 🔗

const responseJson = xml2Json(pm.response.text());//fetch xml data in text/string
const city=responseJson["tns:ROOT"]["tns:Account"][1]["acc:address"]["add:city"];
console.log(city);

In the above case, we have to explicitly define the key to extract data for each element/node, which will not be easy in most cases. As there can be a possibility, prefix changes wherein instead of tns , ns1 comes, etc. Then above logic will correspond to an invalid object.

Another approach is to convert the XML with namespaces to a form where there are no namespaces. The output of XML response as if there are no namespaces involved. For example:

2 For this, we will be using the “xml2js " library in Postman. 🔗

Sample code snippet:

const respBodyXML = pm.response.text();
const parseString = require('xml2js').parseString;
const stripPrefix = require('xml2js').processors.stripPrefix;
parseString(respBodyXML, {
tagNameProcessors: [stripPrefix], //removing prefix
ignoreAttrs: true, //ignoring all attributes including xmlns
explicitArray: false,
explicitRoot: true//including root tag inside the JSON
}, function (err, result) {
console.log(JSON.stringify(result));
});

Console Output:

{"ROOT":{"Account":[{"name":"Name1","id":"123123","address":{"street":"Street1","city":"City1","state":"State1","Country":"Country1"}},{"name":"Name2","id":"124234","address":{"street":"Street2","city":"City2","state":"State2","Country":"Country2"}}]}}

These are some of the ways by which we can handle XML requests inside Postman.

Please share your valuable feedback in the comments below! 😊

Mock a REST Service using SOAPUI with JSON response

Sun, 04 Oct 2020 00:00:00 +0000

Sometimes, backend service is not ready to receive requests and we may want to mock the service. In such cases we have sample request and response but integration cannot be tested unless we actually hit the service. Mocking the services is a good option to continue and progress.

In this tutorial, we will use SOAP UI to mock a REST service with POST method and JSON response(static in nature). This is the basic tutorial to configure static response with no request validation. We can use the SOAP UI to test the created mock service also before, we try to hit from any Integration service.

Sample URL(for actual service): http://localhost:8080/api/test
Sample JSON Response:

{
 "name": "John",
 "age": 30,
 "car": null,
 "regitered": true
}

Steps: 🔗

1. Open SOAP UI and click on REST to create a New REST Project.

2. Provide the url for the api you want to create a mock service.

3. Once the project is created, right click on host name(http://locatlhost:8080) under REST Project 1 and then click on Generate REST Mock Service.

4. Specify the name for mock service and click on OK.

5. A window will open showing that mock service **GET _/api/test _**has been started at port 8089.

6. Click on stop button to stop the mock service. Click on /api/test under REST MockService 1. Change method from GET to POST.

7. Mock response is specified as Response1 which was automatically created. Double click Response 1 from this window or navigator window.

8. Keep HTTP Status Code to be 200 - OK , if your back end application sends the status code 200 else change to the code which backend application sends. Update the sample response under content window editor. Specify media type to application/json.

9. Start the mock service. You will see the mock service POST /api/test is started at port 8089.

10. To test this service, open the request under REST Project 1. Update the hostname:port with the value for mock service. Here in this case will be http://localhost:8089. Click on run.

You will see the response is received. And also we can see in logs that request triggered and is successful.

Note: In SOAP UI, we can also configure dynamic response using groovy script.