SQL/MP DLL Regions

Discussion:

SQL/MP DLL Regions

(too old to reply)

Randall

2020-11-22 18:40:58 UTC

Hi All,

I have a c89 program that is calling a DLL that uses SQL/MP. I'm using dlopen/dlsym to get to the methods. On the first call (happens to be BEGIN WORK), I'm getting SQL [-8025] (not SQL compiled) despite the fact that the SQLCOMP of the DLL works without errors. There is no SQLCOMP for the calling program, which has no SQL region.

Anyone have experience with this situation?

Thanks,
Randall

Keith Dick

2020-11-22 21:58:12 UTC

Permalink

Post by Randall
Hi All,
I have a c89 program that is calling a DLL that uses SQL/MP. I'm using dlopen/dlsym to get to the methods. On the first call (happens to be BEGIN WORK), I'm getting SQL [-8025] (not SQL compiled) despite the fact that the SQLCOMP of the DLL works without errors. There is no SQLCOMP for the calling program, which has no SQL region.
Anyone have experience with this situation?
Thanks,
Randall

The SQL/MP Programming Manual for COBOL says explicitly that SQL/MP statements are not supported in user libraries, SRLs, or DLLs. I did not find a similar restriction in the manual for C, but I'd be very surprised if the same restriction did not apply for C as well.

Keith Dick

2020-11-22 22:13:40 UTC

Permalink

I checked the manual for C again, and it does say that the object file specified to the SQLCOMP command must not be a user library or system library, so that is a somewhat similar restriction, just not mentioning SRLs or DLLs. I don't know why they updated the programming manual for COBOL to document the restriction for SRLs and DLLs, but did not make the same update for the programming manual for C. I am pretty sure that this does NOT mean that SQL/MP statements can be used in a DLL -- I am pretty sure that it is just an omission from the manual.

Randall

2020-11-22 22:21:48 UTC

Permalink

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

Keith Dick

2020-11-22 22:51:32 UTC

Permalink

Post by Randall

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

I don't believe the issue has anything to do with catalogs. If the SQL/MP runtime needs any statically-allocated global memory, that would be a problem for putting SQL/MP code in a user library, and maybe in DLLs, too. I have a dim recollection that SQL/MP does use static global memory for the structures used to pass the host variables for each statement to the SQL runtime, but I might be remembering that wrong.

The other possible challenge is that the runtime has to load the query plans from the SQL region of the object file in which the SQL statements reside. I think finding the name of the object file would be possible, though I'd have to read some manuals to refresh my memory on whether that could be done.

SQL/MX can be used in a DLL, so if you can switch to using SQL/MX in the DLL (SQL/MX can access SQL/MP tables, you know), that might solve your problem.

Randall

2020-11-24 20:01:43 UTC

Permalink

Post by Randall

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

I don't believe the issue has anything to do with catalogs. If the SQL/MP runtime needs any statically-allocated global memory, that would be a problem for putting SQL/MP code in a user library, and maybe in DLLs, too. I have a dim recollection that SQL/MP does use static global memory for the structures used to pass the host variables for each statement to the SQL runtime, but I might be remembering that wrong.
The other possible challenge is that the runtime has to load the query plans from the SQL region of the object file in which the SQL statements reside. I think finding the name of the object file would be possible, though I'd have to read some manuals to refresh my memory on whether that could be done.
SQL/MX can be used in a DLL, so if you can switch to using SQL/MX in the DLL (SQL/MX can access SQL/MP tables, you know), that might solve your problem.

I have a separate implementation for SQL/MX - that part is no issue. Yes, global memory is required, but when I get failures, the issue definitely is highlighting the DLL not being registered even through it is registered in the SQL Catalog. Ah well. I made up something that worked - localized static compile of the SQL module.

Keith Dick

2020-11-24 20:25:01 UTC

Permalink

Post by Randall

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

I don't believe the issue has anything to do with catalogs. If the SQL/MP runtime needs any statically-allocated global memory, that would be a problem for putting SQL/MP code in a user library, and maybe in DLLs, too. I have a dim recollection that SQL/MP does use static global memory for the structures used to pass the host variables for each statement to the SQL runtime, but I might be remembering that wrong.
The other possible challenge is that the runtime has to load the query plans from the SQL region of the object file in which the SQL statements reside. I think finding the name of the object file would be possible, though I'd have to read some manuals to refresh my memory on whether that could be done.
SQL/MX can be used in a DLL, so if you can switch to using SQL/MX in the DLL (SQL/MX can access SQL/MP tables, you know), that might solve your problem.

Have you considered the possibility that the error message could be incorrect? At times I have wasted large amounts of effort because I was misdirected by an error message that was misleading, and sometimes utterly wrong.

I am pretty sure the runtime check does not look in the catalog, but checks the flag in the object file that says the object file is SQL-compiled. Maybe the runtime check doesn't know how to find the DLL object file and only looks at the main object file. Perhaps you could trick it by including an unused SQL/MP statement in the main object file and SQL-compile the main object file. I don't know that would work. I think you'd still have trouble with the global memory that SQL/MP uses, but I might be wrong about that.

Since you say you have SQL/MX working in a DLL, and since SQL/MX can access SQL/MP data, I kind of wonder why you would continue trying to get SQL/MP to work in a DLL. What benefit would that give you?

Keith Dick

2020-11-24 20:28:52 UTC

Permalink

Post by Randall

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

I don't believe the issue has anything to do with catalogs. If the SQL/MP runtime needs any statically-allocated global memory, that would be a problem for putting SQL/MP code in a user library, and maybe in DLLs, too. I have a dim recollection that SQL/MP does use static global memory for the structures used to pass the host variables for each statement to the SQL runtime, but I might be remembering that wrong.
The other possible challenge is that the runtime has to load the query plans from the SQL region of the object file in which the SQL statements reside. I think finding the name of the object file would be possible, though I'd have to read some manuals to refresh my memory on whether that could be done.
SQL/MX can be used in a DLL, so if you can switch to using SQL/MX in the DLL (SQL/MX can access SQL/MP tables, you know), that might solve your problem.

Have you considered the possibility that the error message could be incorrect? At times I have wasted large amounts of effort because I was misdirected by an error message that was misleading, and sometimes utterly wrong.
I am pretty sure the runtime check does not look in the catalog, but checks the flag in the object file that says the object file is SQL-compiled. Maybe the runtime check doesn't know how to find the DLL object file and only looks at the main object file. Perhaps you could trick it by including an unused SQL/MP statement in the main object file and SQL-compile the main object file. I don't know that would work. I think you'd still have trouble with the global memory that SQL/MP uses, but I might be wrong about that.
Since you say you have SQL/MX working in a DLL, and since SQL/MX can access SQL/MP data, I kind of wonder why you would continue trying to get SQL/MP to work in a DLL. What benefit would that give you?

Or maybe the SQL-compiled flag in the object file gets copied into the PFS when the process gets started and the SQL/MP runtime check looks at the copy in the PFS. It probably wouldn't copy a SQL-compiled flag from a DLL into the PFS.

Randall

2020-11-25 19:18:07 UTC

Permalink

Post by Randall

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

I don't believe the issue has anything to do with catalogs. If the SQL/MP runtime needs any statically-allocated global memory, that would be a problem for putting SQL/MP code in a user library, and maybe in DLLs, too. I have a dim recollection that SQL/MP does use static global memory for the structures used to pass the host variables for each statement to the SQL runtime, but I might be remembering that wrong.
The other possible challenge is that the runtime has to load the query plans from the SQL region of the object file in which the SQL statements reside. I think finding the name of the object file would be possible, though I'd have to read some manuals to refresh my memory on whether that could be done.
SQL/MX can be used in a DLL, so if you can switch to using SQL/MX in the DLL (SQL/MX can access SQL/MP tables, you know), that might solve your problem.

Have you considered the possibility that the error message could be incorrect? At times I have wasted large amounts of effort because I was misdirected by an error message that was misleading, and sometimes utterly wrong.
I am pretty sure the runtime check does not look in the catalog, but checks the flag in the object file that says the object file is SQL-compiled. Maybe the runtime check doesn't know how to find the DLL object file and only looks at the main object file. Perhaps you could trick it by including an unused SQL/MP statement in the main object file and SQL-compile the main object file. I don't know that would work. I think you'd still have trouble with the global memory that SQL/MP uses, but I might be wrong about that.
Since you say you have SQL/MX working in a DLL, and since SQL/MX can access SQL/MP data, I kind of wonder why you would continue trying to get SQL/MP to work in a DLL. What benefit would that give you?

I think the SQL-valid flag is more of relevance to the PFS since it is mutable. Since there isn't one when the program is loaded, when the DLL is dynamically loaded, there is nowhere for the flag to be placed. Perhaps not using dlopen might have helped, but I've got it working using a static link.

Richard Carr

2020-11-26 20:24:14 UTC

Permalink

This is a very old topic and your GNSC SQL/MP analyst would have known the answer right away.

The SQL/MP executor simply looks in the PROGRAM object file for the compiled SQL region. It has no code that looks in the UL/DLL object file.

It's pretty brain-dead that SQLCOMP will compile the SQL source in a UL/DLL when there is no way that could be used by the SQL executor.

There was a special unpublished capability to ask SQL to load SQL object from any random object file; this was used for a product that allowed a X86 program to perform SQL statements on a PC, which were then sent to the NonStop for actual execution.

Randall

2020-11-25 19:15:58 UTC

Permalink

Post by Randall

Well, all I can say is "drats". That would be a very nice feature to have. But I can see the loader having issues trying to chase down whether DLLs are registered in the proper catalog.

I don't believe the issue has anything to do with catalogs. If the SQL/MP runtime needs any statically-allocated global memory, that would be a problem for putting SQL/MP code in a user library, and maybe in DLLs, too. I have a dim recollection that SQL/MP does use static global memory for the structures used to pass the host variables for each statement to the SQL runtime, but I might be remembering that wrong.
The other possible challenge is that the runtime has to load the query plans from the SQL region of the object file in which the SQL statements reside. I think finding the name of the object file would be possible, though I'd have to read some manuals to refresh my memory on whether that could be done.
SQL/MX can be used in a DLL, so if you can switch to using SQL/MX in the DLL (SQL/MX can access SQL/MP tables, you know), that might solve your problem.

Have you considered the possibility that the error message could be incorrect? At times I have wasted large amounts of effort because I was misdirected by an error message that was misleading, and sometimes utterly wrong.
I am pretty sure the runtime check does not look in the catalog, but checks the flag in the object file that says the object file is SQL-compiled. Maybe the runtime check doesn't know how to find the DLL object file and only looks at the main object file. Perhaps you could trick it by including an unused SQL/MP statement in the main object file and SQL-compile the main object file. I don't know that would work. I think you'd still have trouble with the global memory that SQL/MP uses, but I might be wrong about that.
Since you say you have SQL/MX working in a DLL, and since SQL/MX can access SQL/MP data, I kind of wonder why you would continue trying to get SQL/MP to work in a DLL. What benefit would that give you?

It's a requirement thing for product capabilities. I need to be able to access SQL/MP content for customers who are not 100% guaranteed to have SQL/MX. I would rather have gone down the SQL/MX path in the first place (because I know it rather intimately). Such is life. The requirement was to have a completely non-SQL main with product infrastructure that could be shipped in binary, with an SQL/MP module that could be built with whatever RVU the customer has. I wanted to do that with a DLL, but alas, that didn't work out - it probably was the loader not checking the file name in question in the catalog - not an issue in SQL/MX. In any event, I solved it by avoiding DLLs altogether. It's a bit more rigid, but does get the job done correctly and runs on any HPE supported RVU so I'm good, without my having to maintain different SQL/MP instances of those RVUs - this would be a great use case for vNonStop if you ask me.