One-More-Query on DecipherMiddleware

Playing With JSON data in Oracle Database- Part1

Tue, 05 Dec 2023 19:43:21 +0000

Whenever we hear Oracle Database, we imagine storing data in rows and columns. The data stored in cells generally is in primitive data types(char, varchar2, int, date, etc.). What if we need to store JSON data in the table. The first thing that may come into our minds is to convert JSON data to string(varchar2) or have data as clob/blob and insert it into the table. But with this if we need to query a particular field or update it, tricky one. This blog is a part of a series. The scope includes, creation of table having column with JSON datatype, insertion of JSON data and querying the JSON data.

Umm.. How to do? 🔗

Let’s put the developer hat on! and start exploring. In this blog, Oracle Database 23c free is used.

Create a table with a column to store JSON data 🔗

JSON data can be used natively with RDBMS features in Oracle Database 23c. Let’s start the journey by creating a simple database table for storing JSON data.

Create a sequence SEQ_PO for the id column.

create sequence SEQ_PO start with 1000 increment by 1 NOCACHE NOMAXVALUE;

Create a table q_purchaseorder to store purchase order documents in JSON format.

CREATE TABLE q_purchaseorder
(id NUMBER DEFAULT ON NULL SEQ_PO.NEXTVAL PRIMARY KEY,
date_loaded TIMESTAMP (6) WITH TIME ZONE,
 po_document JSON);

If, we want to store the JSON data in Varchar2, then we need to add a constraint “ensure_json” to check if the data is in JSON format. For example:

CREATE TABLE q_purchaseorder
(id NUMBER DEFAULT ON NULL SEQ_PO.NEXTVAL PRIMARY KEY,
date_loaded TIMESTAMP (6) WITH TIME ZONE,
po_document VARCHAR2 (23767)
CONSTRAINT ensure_json CHECK (po_document is json));

The table q_purchaseorder is created.

Insert data to the JSON column 🔗

Let’s insert some purchase orders JSON data to the newly created table in po_document column.

INSERT INTO q_purchaseorder (date_loaded,po_document) VALUES (
 SYSTIMESTAMP, '{
 "pONumber": "1001",
 "reference": "dm-1001",
 "requestor": "One More Query",
 "user": "dm_1",
 "costCenter": "CC1",
 "shippingDetails": {
 "name": "Test 1",
 "address": {
 "street": "Street 1",
 "city": "Amritsar",
 "state": "Punjab",
 "zipCode": "143001",
 "country": "India"
 },
 "phone": "+91-99999-99999"
 },
 "message": "Courier",
 "lineItems": [
 {
 "itemNumber": 1,
 "part": {
 "description": "Test product1",
 "unitPrice": 200,
 "uPCCode": "123456789"
 },
 "quantity": 8.0
 },
 {
 "itemNumber": 2,
 "part": {
 "description": "Test product2",
 "unitPrice": 250,
 "uPCCode": "987654321"
 },
 "quantity": 5
 }
 ]
}'
);

The above JSON record gets inserted into the table.

select * from q_purchaseorder;

Query the JSON data 🔗

The easiest way to query the JSON data is to use dot notations. The JSON data in the column must be well formed. If the data type is not JSON, varchar2, clob/blob column must have “is json “ constraint. Taking the data inserted in the previous step as an example, below are the various queries to extract values from different JSON fields.

Query the PO number. 🔗

SELECT po.PO_DOCUMENT.pONumber FROM q_purchaseorder po;

Query the unit price for second line item. 🔗

SELECT po.PO_DOCUMENT.lineItems[1].part.unitPrice FROM q_purchaseorder po;

The above two examples output the data as JSON object instead of SQL number or SQL STRING, as the column datatype is JSON. If datatype of the column is not JSON, the query will give a result in VARCHAR2 format. An item method, such as number(), string(), etc., can convert JSON/VARCHAR2 output from the query into the desired format. For example:

Query the quantity for the second line item and output in number format. 🔗

SELECT po.PO_DOCUMENT.lineItems[0].quantity.number() FROM q_purchaseorder po where po.id=1000;;

Another way to query data is to use JSON_VALUE SQL function. For example:

Query the PO Number in number format using JSON_VALUE SQL function. 🔗

SELECT JSON_VALUE(po_document, '$.pONumber.number()') from q_purchaseorder where po.id=1000;;

Pretty simple, yeah? Let’s make it a bit more complex.

How to fetch an item description from purchase order JSON where unit price is less than 225?

Scratching the head!!!! Let’s use json_query sql function along with filter conditions to accomplish the task.

SELECT json_query(po.po_document, '$.lineItems[*].part?(@.unitPrice < 225).description' with ARRAY wrapper)
FROM q_purchaseorder po
WHERE po.id=1000;

WITH ARRAY WRAPPER is used to wrap the results in array, as output may contain more than one values.

References: 🔗

Please feel free to share your valuable feedback ! 😊😊😊

Transactional Event queues in Oracle Database 23c Free (PL/SQL)

Thu, 31 Aug 2023 17:42:45 +0000

The Oracle Database 23c provides integrated, robust and feature-rich queuing systems AQ (Advance Queues) and TxEventQ(Transactional Event Queues). Oracle Transactional Event Queues(TxEventQ) facilitate the implementation of event-based/streaming applications. TxEventQ can be integrated with open source event streaming platform Kafka.

So, let’s wear our developer’s hat and create a simple TxEventQ in Oracle Database 23c.

Pre-Requisites 🔗

Oracle Database 23c free
User with SYSDBA privileges

Steps 🔗

Create a database user and grant privileges. Login using sysdba user and execute the below commands using SQL developer or SQL plus.

-- CREATE USER <USERNAME> IDENTIFIED BY <PASSWORD>
CREATE USER dm_teq IDENTIFIED BY *****;
-- GRANT ACCESS AND ROLES TO THE USER
GRANT CONNECT, RESOURCE, AQ_ADMINISTRATOR_ROLE TO dm_teq;
GRANT EXECUTE ON DBMS_AQ TO dm_teq;
GRANT EXECUTE ON DBMS_AQADM TO dm_teq;
-- OPTIONAL GRANT TABLESPACE TO THE USER
GRANT UNLIMITED TABLESPACE TO dm_teq;

There is a procedure CREATE_TRANSACTIONAL_EVENT_QUEUE available under package DBMS_AQADM that can be used to create TxEventQ.

PROCEDURE CREATE_TRANSACTIONAL_EVENT_QUEUE (
 queue_name IN VARCHAR2,
 storage_clause IN VARCHAR2 DEFAULT NULL,
 multiple_consumers IN BOOLEAN DEFAULT FALSE,
 max_retries IN NUMBER DEFAULT NULL,
 comment IN VARCHAR2 DEFAULT NULL,
 queue_payload_type IN VARCHAR2 DEFAULT JMS_TYPE,
 queue_properties IN QUEUE_PROPS_T DEFAULT NULL,
 replication_mode IN BINARY_INTEGER DEFAULT NONE);

BEGIN
 dbms_aqadm.create_transactional_event_queue(queue_name => 'demo_topic', multiple_consumers => true, queue_payload_type => dbms_aqadm.jms_type
 );
-- Start the
 dbms_aqadm.start_queue(queue_name => 'demo_topic');
END;

The above command will create a topic(with multiple consumers) with the name “demo-topic”, payload type as JMS and start the topic.

Fetch TxEventQ Params 🔗

Now, the topic is created. What if we need to fetch queue/parameters, we can use the get_queue_parameter procedure as below.

set SERVEROUTPUT on;
declare
 out_var VARCHAR2(100);
begin

 dbms_aqadm.get_queue_parameter(
 queue_name=> 'demon_topic',
 param_name=> 'SHARD_NUM',
 param_value=> out_var
 );
 dbms_output.put_line(out_var);
end;

Output: Below output will display the number of partitions for the topic.

PL/SQL procedure successfully completed.

Add a Subscriber to the TxEventQ 🔗

Execute below pl/sql block to add a subscriber to the demo_topic(TxEventQ).

DECLARE
 subscriber SYS.aq$_agent;
 /* Add subscribers omq to the suscriber list: */
BEGIN
 subscriber := SYS.aq$_agent('OMQ', NULL, NULL);
 DBMS_AQADM.ADD_SUBSCRIBER(queue_name => 'demo_topic',
 subscriber => subscriber);
END;

Query the list of subscribers for the topic.

-- view name : aq$<queue_name>_s
select * from aq$demo_topic_s;

Output:

Publish a message on the topic 🔗

--- config: theme: 'neutral' --- flowchart LR A("Publisher") --> id1[(demo_topic)]

We have created a topic based on JMS Message type. Payload should also be of type JMS message.

DECLARE
 enqueue_options dbms_aq.enqueue_options_t;
 message_properties dbms_aq.message_properties_t;
 message_handle RAW(16);
 message SYS.AQ$_JMS_MESSAGE;
BEGIN
 message:= SYS.AQ$_JMS_MESSAGE.construct(DBMS_AQ.JMS_TEXT_MESSAGE);
 message.set_text('hello world');
 dbms_aq.enqueue(queue_name => 'demo_topic', enqueue_options => enqueue_options, message_properties => message_properties, payload => message
 , msgid => message_handle);
DBMS_OUTPUT.PUT_LINE(message_handle);
END;

The message has been published on the topic, let’s validate. We can check the status of the message from the available view “aq$<queue_name>”. Let’s run the below query to know the status of the message.

select msg_id, user_data,msg_state,enq_timestamp,deq_timestamp from aq$demo_topic;

Output:

--- config: theme: 'neutral' --- flowchart LR A("Publisher") --> db[(demo_topic)] db --> B(subscriber OMQ)

Let’s use the subscriber created in the earlier step to extract the message from the topic.

DECLARE
 dequeue_options dbms_aq.dequeue_options_t;
 message_properties dbms_aq.message_properties_t;
 message sys.aq$_jms_message;
 message_handle RAW(16);
 message_text VARCHAR2(100);
BEGIN
dequeue_options.navigation := DBMS_AQ.FIRST_MESSAGE;
dequeue_options.consumer_name := 'omq';
 dbms_aq.dequeue(queue_name => 'demo_topic', dequeue_options => dequeue_options, message_properties => message_properties, payload => message
 , msgid => message_handle);

 message.get_text(message_text);
 dbms_output.put_line(message_text);
END;

Output :

hello world
PL/SQL procedure successfully completed.

This is a simple example to dequeue a message from TxEventQ. Let’s run the below query to know the status of the message.

select msg_id, user_data,msg_state,enq_timestamp,deq_timestamp from aq$demo_topic;

The state now changed from READY to PROCESSED.

This is a simple walkthrough to configure and use TxEventQ in Oracle Database 23c with the help of PL/SQL.

Please share your valuable feedback! 😄😄😄

One-More-Query on DecipherMiddleware

Playing With JSON data in Oracle Database- Part1

Umm.. How to do? 🔗

Create a table with a column to store JSON data 🔗

Insert data to the JSON column 🔗

Query the JSON data 🔗

Query the PO number. 🔗

Query the unit price for second line item. 🔗

Query the quantity for the second line item and output in number format. 🔗

Query the PO Number in number format using JSON_VALUE SQL function. 🔗

References: 🔗

Transactional Event queues in Oracle Database 23c Free (PL/SQL)

Pre-Requisites 🔗

Steps 🔗

Fetch TxEventQ Params 🔗

Add a Subscriber to the TxEventQ 🔗

Publish a message on the topic 🔗

Subscribe messages from the topic 🔗

References: 🔗