I should make some observations about performance analysis and try to group them into some logical areas.

Firstly you need to quantify performance against a known benchmark or your own benchmark.  In general, everyone’s ERP is different so the most beneficial comparisons will come from your own benchmarks.  Performance can generally ALWAYS be improved.  It can be improved with hardware, software, code, data or process changes or a combination of the 5.

Where do performance problems commonly start from:

This is a difficult question to answer – but, I’ll provide a list and some examples

• database – this is generally the cause of many performance issues, especially for an ERP like JD Edwards.  JD Edwards is database agnostic, which means that it’s queries and interface to the database are written generically.  This is fantastic if you ever want to change database platform, but it comes at a price.  The price is specific ability to implement database tuning on a platform by platform basis.  Don’t stop reading now, there is more.
  • Each database that is supported on JD Edwards gives you the ability to tune queries outside of the ERP, so this is not a limitation of the ERP – it’s how you manage it.  For example on an oracle database you cannot implement hints via the oci based API’s that JD Edwards provide you with.  But, you can implement a database profile to effect the same changes.
  • All databases allow you to look at poor performing statements and address them with various panacea’s.  Indexes, etc.
  • Size of data.  This is critical, I always recommend to maintain the database size and then you’ll not be continuing to tune – as everything will be in a nice performance equilibrium.   Statistics won’t need to change – everything will be running sweetly.
  • Indexing in general.  Look at your read to write ratio’s, generally a new index is not going to hurt you.  If you are worried about updating indexes and the time it takes (you should), then look at what indexes have not been used over the last 6 months and drop them (yes – you heard me).
• Hardware allocations.  this is the most basic problem with the most basic fix (unless you have all 128 cores allocated and you still need more power).  This is simple to diagnose and simple to fix in their virtual world – look at the stats and allocate more where needed.  More memory on the DB server, bring the data closer to the CPU!
• Software tuning.  Generally, this is where all of the action is going to occur and where you are going to get a lot of gains.  This can be changes to code, changes to process or changes to configuration.  Look at the classic bug with urnandom that I posted about earlier.  This will bring a site to it’s knees, and you’ll spend weeks trying to find it.  I’ve seen some of the biggest improvements come from process change, where the change of a PO (for example) get’s the software to skip a certain unrequired step and turn it all around. 
• network, this is becoming less critical given the 1GB and 10GB networks that are being seen at the moment.  WANs are getting better and network prioritisation is getting better, so I see less and less of this being problematic.

What tools are available to you to fix performance issues in JD Edwards?

Native:

Careful analysis of batch performance (using queries like have been identified in the link.

Careful analysis of BSFN timings that you can extract from Server Manager are great for working out if there are particular BSFNs that are slow.

Performance Workbench:

download the Myriad tool that gives sub second time stamps on JD Edwards database and web functions.  This benchmarks performance and allows comparison graphing with old results.  Myriad will even send you information on how you stack up with industry standards – how your site compares!  Myriad can tell you were you rank in terms of the many clients that have ran the software.

ERP Analytics:

Gives you unparalleled insight into performance and end user activity in your ERP.  Who is doing what and when.  Ever wanted to see average performance with a geo overlay – easy. 

Average page load time: The average amount of time (in seconds) it takes that page to load, from initiation of the pageview (e.g., click on a page link) to load completion in the browser.

Average server response time:  The time for your server to respond to a user request, including the network time from the user’s location to your server

We track all these metrics and more and allow you to view them by application, user, region (pending network geography and more).

Remember to see here for more information.

Load Testing as a Service:

Actually load test JD Edwards with specific oracle tools and specific JD Edwards load testing knowledge.  Get Myriad to load test your ERP with JD Edwards technical experts, people who know how fast JD Edwards can go and how to make improvements that make the ERP faster.

JD Edwards load testing

This should help a lot of people.  This is a simple vbs script that you can schedule every 15 minutes (or when you want), which will tell you if your specified errors have been logged in a JD Edwards log file.  Note that this script is completely generic and you do not need to use it for JD Edwards.

Schedule it every 15 minutes and you are away.  If you schedule longer or shorter, change the scriptSleepTime variable in the header.

dim filesys
set filesys=CreateObject("Scripting.FileSystemObject")
Set objFSO = CreateObject("Scripting.FileSystemObject")
Dim objShell 'instance of the wshSHell object
set objShell = CreateObject("WScript.Shell")
FromEmailAddress="jde@myriad-it.com"
Const strSmartHost = "smtp"
Company="Myriad IT"
RecipientNames = ARRAY("smoir@myriad-it.com",someone@thing.com)
ScriptSleepTimeMinutes=15


strSearchForArray = ARRAY("ORA-03114","ORA-03113","GetRecipientPreferences")
Set oFSO = CreateObject("Scripting.FileSystemObject")
JDELogFolder = "D:\JDEdwards\E910\log"

Set objFolder = objFSO.GetFolder(JDELogFolder)

Set jdeLogFiles = objFolder.Files
For Each objFile in jdeLogFiles
'Wscript.Echo objFile.Name
    strFile = JDELogFolder & "\"& objFile.Name
    set objFile = objFSO.getFile(strFile)
    if objFile.size > 0 then
      for each SearchString in strSearchForArray 
        If InStr(oFSO.OpenTextFile(strFile).ReadAll, SearchString ) > 0 Then
'Email Details
            if objFile.size < 2000000 then
              If DateDiff("n", objFile.DateLastModified, Now) < ScriptSleepTimeMinutes Then
'File has been modified in past 10 minutes.
                i=sendMail("URGENT: "& SearchString  & " Found in log file - see attached", "See attachment", strFile)
              End If
                    else
                                i=sendMail("URGENT: "& SearchString  & " Found in log file - too big to attach", "NO attachment", "")  
            end if
        END If
      next
    END If
Next
wscript.echo "Script complete"


'#####################################################################
'FUNCTION:sendMail
'#####################################################################
function sendMail(subjectText, emailBody, attachment)
  on error resume next
  Dim iMsg    
  Dim iConf    
  Dim Flds    
  Const cdoSendUsingPort = 2    

  for each name in RecipientNames
'Create the message object    
    Set iMsg = CreateObject("CDO.Message")       
'Create the configuration object    
    Set iConf = iMsg.Configuration        
'Set the fields of the configuration object to send using SMTP via port 25.    
    With iConf.Fields       
      .item("http://schemas.microsoft.com/cdo/configuration/sendusing") = cdoSendUsingPort
      .item("http://schemas.microsoft.com/cdo/configuration/smtpserver") = strSmartHost
      .Update    
    End With   

'Set the message to,from,subject,body properties.    
    if strlen(attachment) > 1 then
      With iMsg
        .AddAttachment attachment
        .To = name    
        .From = FromEmailAddress       
        .Subject = subjectText &  ""& now()       
        .TextBody = chr(1)  & ""& emailBody       
        .Send    
      End With    
    else
      With iMsg
        .To = name    
        .From = FromEmailAddress       
        .Subject = subjectText  & ""& now()       
        .TextBody = emailBody       
        .Send    
      End With    
    end if
    set iMsg = Nothing 
  next

end function

This is potentially one of the best posts that I’ve done in a while… Well, perhaps it might be one of the most helpful ones in a while.  But, I’d best get off my soap box and get into the nitty gritty of the post.

It’s a common situation that a UBE goes into Error sometimes and works the rest of the time…  So, you need debug…  Your choices are limited for enabling logging, as it’s pretty global on your enterprise server and will be enabled for all UBE’s and all processes that start after you’ve enabled the setting in the JDE.INI file.

I had this conundrum at a client recently and decided to try and enable logging in an OSA.  If I could enable logging with an OSA, I could map this to any UBE that I needed to enable logging for.  Therefore, no package builds, no deploys, no global logging enabled – just logging for the UBE’s that I want to log WHEN I want to log them. 

So now you can configure what logging is enabled for a UBE based on :

• version
• environment
• machine

Too good to be true, no – it actually worked!  The best thing is, I’m going to give you all of the source code as well.  All 1 line of it!

EXTERNC APIEXPORT void CDECL myriadEnableLogging(POSA_REPORT_INFO pOSAReportInfo,
POSA_LINK_INFO pOSALinkInfo,
unsigned long ulNumberOfLinks)
{
	//jdeDebugInit();
	changeLogSettingsNoINI(_J("Output"), 1);
}

 

So you can map this to the UBE that you want logging via the OSA application.

You need to compile the code above into a DLL (in my case called myOSA) and plonk this in the enterprise server bin32 dir. 

Then create the OSA definition P986168 for enableLogging

The first button above and then add

Then map your UBE’s that you want logging on for (the second button)

The second button

So in this example I’m enabling logging for all users that run R42565 in JDV910.

If you want a windows DLL prebuilt with this code, let me know and I’ll probably be nice and send it to you.

I get asked this all of the time, so thought that I’d add a couple of slides.  Note that do not enter into any contracts based upon this information, please see your friendly oracle people for the gospel!

Apps:

9.1 2012

9.X 2015

9.Y 2018

Every 3 years an app release

Tools

Major tools releases (9.1, 9.2 etc) just before the application release – on a three year cycle.

A significant dot release for the tools every 6 months or so.

Once again, this is a community service announcement, because I can never find the latest edition of this support matrix.

What really scares me is the release dates of each release, because I’ve been working with JDE through ALL of them…  <panic>Oh no, I’m old.</panic>

This is very handy for determining what release is supported until when.  What is really important is that JD Edwards has a published roadmap until at least 2021 + 5 years of support – 2026!  That makes it an awesome investment in the future with guaranteed support from Oracle during this time.

Please, always go to the horses mouth (so to speak) to ratify anything regarding support and releases. 

JD Edwards current published roadmap

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1317204.1&_afrLoop=417429168425373&_afrWindowMode=0&_afrWindowId=null

For all oracle applications and their support and release matrix:

http://www.oracle.com/us/support/library/lifetime-support-applications-069216.pdf

I know I’ve said it before, but I’m going in again.

I have complete find rage with windows.

findstr /c:”10.32.8.40” finds heaps of files and heaps of instances in the jde log directory.

Find window, searching in file contents – finds nothing.

Don’t EVER rely on windows find to have your back.

I’m using findstr because the find window is complete rubbish, but now I have another issue.  I want to list the files I’ve “found” in explorer to copy them to another folder.  Ironically I want to zip them up and move them to my Mac so that I can sed and awk the cr@p out of them and get to the bottom of a tricky network problem at a client.

So, I have the list of files in a dos window and I paste them into EXCEL and add some “ OR “ with a basic concatenate function. (& thanks Shae) [=+A4&" OR "].  Then I past then into the explorer find window and prefix it with the ultra intuitive “system.filename:”

The exact syntax for the find multiple and specific files:

system.filename: jde_11156.log OR     jde_12268.log OR     jde_3680.log OR     jde_4296.log OR     jde_4516.log OR     jde_4596.log OR     jde_5384.log OR     jde_7776.log OR     jde_7896.log OR     jde_9220.log OR     jde_9772.log OR     jde_9980.log

So now, I have a window with all of the files that contains the text I’m searching for without crappo windows find.

Quickly copy and create zip and send to unix and now I can get to work.  cygwin do I hear you calling – maybe you are right.

I’m tracing through JD Edwards log files and am searching for an IP address that is having connection issues in the jdenet log files.  They are easily identifiable by grepping for my problem IP address:

>grep 10.32.8.40 * |more

jde_3680.log:   10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer
jde_3680.log:   10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer
jde_3680.log:   10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer
jde_3680.log:   10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer
jde_3680.log:   10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer
jde_3680.log:   10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer

But I need the timestamp, so I want the previous line in the log file, oh wait – we are on a totally legendary OS like unix or linux and this is massively easy:

>grep -B1 10.32.8.40 * |more

jde_11156.log-11156/11692 MAIN_THREAD                           Sun Feb 01 01:04:11.728000      NETRCV.C443
jde_11156.log:  10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer
--
jde_11156.log-11156/11692 MAIN_THREAD                           Mon Feb 02 23:00:42.996000      NETRCV.C443
jde_11156.log:  10054-Error from Host=<10.32.8.40> in recv returned 10054 (WSAECONNRESET): Connection was reset by peer

Wow, so now I can start reconciling the timestamps and the errors.  Fantastic!

-B num, --before-context=num
        Print num lines of leading context before each match.  See also the -A and -C options.

If you’ve got lost of data you might have got to this post.  If you are attempting to go live is some sort of second rate cloud, you might have come to this page.  Let me tell you a story and highlight things with some numbers.

Once upon a time there was a table in JD Edwards, let’s call it F03B11 that had 46 million rows.  This table took 5.5 hours to convert to unicode using the JD Edwards standard unicode conversion methodology.

A new conversion method got this down to 15 minutes, but what changed?  Everything!

Read on for the details:

The initial timings were attained during an upgrade and platform migration to the cloud.  This poor client was moving from the stability and performance of their impressively specified p-Series to their no so impressive (at first) cloud provider.

This is an oracle based system, both pSeries and going to cloud. 

I guess this turns out to be a bit of a hard core comparison of cloud vs. physical tin that you can touch and also big unix vs x86 and fibre attached SAN vs – who knows what you are getting in the cloud – 100MBs NAS?

But, you are stuck with your hardware.  You might choose where you are going to run certain items, but in general this piece if fixed.

What can you change to make the process quicker?

Perhaps something like:

1. create table F0911_UNICODE (…. NCHAR NCHAR blah blah)
2. drop all F0911 indexes
3. alter table proddta.f0911 drop constraint F0911_PK;
4. insert into proddta.f0911_UNICODE select * proddta.f0911;
5. commit
6. create all F0911 indexes

This is instead of using the stored procedure and function that JD Edwards calls.

I’ve seen up to 25% improvement (and more) running this manually on the large tables.

It’s easy to get the unicode definitions of the tables, just generate them against a unicode data source (control tables is handy) and then grab the details from SQLDeveloper.

I generally employee the above for the top 20 tables and use the standard conversions for the others.

As a side note, the data in this 46 million row F03B11 was 42.6GB single single byte mode but 71.5GB in unicode.  For this table it was 59% greater.

You can probably tell from my last post (not the last post) that I’m working on making a unicode conversion go a lot faster.  There are heaps of ways of doing this, I’m lucky enough to be working on a large oracle site with EE and heaps of CPU’s.  I’m able to give the thing a good thrashing too:

So at the end of the day I need to write 2 different bits of code, 1 to convert all of the huge tables manually, and another to convert the rest of the table with the standard OWTBLCONV.  That is the name of the procedure that oracle gives you to do the conversions for you.  You can just look at this in SQLDeveloper and see what it does, it’s pretty cool.

Anyway, so I’m in unix land and I need a script that will go through all of the tables with CHAR fields and convert them.  Okay, lets get a list:

select  table_name from all_tables t1 where
exists (select 1 from all_tab_columns t2
where t2.data_type = 'CHAR'
and t1.table_name = t2.table_name
and t1.owner = t2.owner)
and t1.owner = 'PRODDTA' ;

Right, that was easy, now which are the big tables again…  Oh yeah, that’s easy:

select segment_name, sum(bytes) from dba_segments where owner = 'PRODDTA' group by segment_name order by 2 desc;

Right we are getting somewhere.  Now there are 4000 tables (approx) in proddta that need to be unicode, so I have a file in unix with 4000 lines.  I want to make my script multi-threaded, so all I do is use the most awesomely awesome command – split:

split -l 500 tables.list parallel_table

So this gives me as many files as I need, starting with “parallel_table” out of my list of tables “tables.list”.  Yes, that easy:

-rw-r--r--    1 moirs    staff          4138 Feb 09 18:42 parallel_tableaa
-rw-r--r--    1 moirs    staff          4139 Feb 09 18:42 parallel_tableab
-rw-r--r--    1 moirs    staff          4079 Feb 09 18:42 parallel_tableac
-rw-r--r--    1 moirs    staff          3688 Feb 09 18:42 parallel_tablead
-rw-r--r--    1 moirs    staff          3619 Feb 09 18:42 parallel_tableae
-rw-r--r--    1 moirs    staff          3640 Feb 09 18:42 parallel_tableaf
-rw-r--r--    1 moirs    staff          3784 Feb 09 18:42 parallel_tableag
-rw-r--r--    1 moirs    staff          3189 Feb 09 18:42 parallel_tableah

That is so Rad!

I then have a script that I call:

#!/usr/bin/ksh
#mkdir ~/uniconvlogs
if [ $# -ne 1 ]
  then
    echo "USAGE $0 <input file containing list of tables>"
    return 1
fi
fileinput=$1
if [ ! -f $fileinput ]
  then
    echo "ERROR:  File $1 does not exist"
    return 1
fi
for filename in `cat $fileinput`
  do
    echo processing $filename
    echo "spool /home/moirs/uniconvlogs/$filename.log">~/uniconvlogs/$filename.sql
    echo "set echo on">> ~/uniconvlogs/$filename.sql
    echo "set feedback on">>~/uniconvlogs/$filename.sql
    echo "set timing on">>~/uniconvlogs/$filename.sql
    echo "set pagesize 8000">>~/uniconvlogs/$filename.sql
    echo "set linesize 8000">>~/uniconvlogs/$filename.sql
    echo "set trimspool on">>~/uniconvlogs/$filename.sql
    echo "alter session force parallel DDL PARALLEL 8;">>~/uniconvlogs/$filename.sql
    echo "alter session force parallel DML PARALLEL 8;">>~/uniconvlogs/$filename.sql
    echo "alter session force parallel QUERY PARALLEL 8;">>~/uniconvlogs/$filename.sql

    echo "select to_char(sysdate,'HH24:MI:SS') , to_char(sysdate,'YYY MM DD') from dual;">>~/uniconvlogs/$filename.sql
    echo "Call ProdDta.OWTBLCONV('PRODDTA', 'J@ck@ss','PRODDTA','PRODDTA', '$filename',0,0);">>~/uniconvlogs/$filename.sql
    echo "spool off">>~/uniconvlogs/$filename.sql
    echo "exit">>~/uniconvlogs/$filename.sql

    sqlplus proddta/J@ck@ss@sp2jde @/home/moirs/uniconvlogs/$filename.sql

done

And a script that does the multi-threading:

#!/usr/bin/ksh
for tablelist in `ls ~/parallel_tablea*`
  do
    convertTables.ksh $tablelist &
done

So now I have a complete suite of rapid conversions with 6 threads.  This is blasting through the workload.

I’ve already taken out the LARGE tables that I’m doing manually, they are done with a script like:

column filename new_val filename ;
select './UnicodeConv/F0101_unicode_conversion_' || to_char(sysdate, 'yyyymmdd' ) ||'.log' filename from dual;
spool &filename

set echo on
set feedback on
set timing on
set pagesize 8000
set linesize 8000
set trimspool on
select to_char(sysdate,'HH24:MI:SS') from dual;
select to_char(sysdate,'YYY MM DD') from dual;


alter session force parallel DDL PARALLEL 8;
alter session force parallel DML PARALLEL 8;
alter session force parallel QUERY PARALLEL 8;

alter table proddta.F0101 rename to F0101_NONUNI;
alter table proddta.F0101_NONUNI drop constraint F0101_PK ;

--stupid, but still uses old name for delete.
--although an index could be called mittens, does not need to be called the table name

DROP INDEX PRODDTA.F0101_0;
DROP INDEX PRODDTA.F0101_10;
DROP INDEX PRODDTA.F0101_11;
DROP INDEX PRODDTA.F0101_12;
DROP INDEX PRODDTA.F0101_13;
DROP INDEX PRODDTA.F0101_14;
DROP INDEX PRODDTA.F0101_15;
DROP INDEX PRODDTA.F0101_16;
DROP INDEX PRODDTA.F0101_2;
DROP INDEX PRODDTA.F0101_3;
DROP INDEX PRODDTA.F0101_4;
DROP INDEX PRODDTA.F0101_5;
DROP INDEX PRODDTA.F0101_6;
DROP INDEX PRODDTA.F0101_7;
DROP INDEX PRODDTA.F0101_8;
DROP INDEX PRODDTA.F0101_9;
DROP INDEX PRODDTA.F0101_CUST_IDX01;

 CREATE TABLE "PRODDTA"."F0101"
   (    "ABAN8" NUMBER,
"ABALKY" NCHAR(20),
"ABTAX" NCHAR(20),
"ABALPH" NCHAR(40),
"ABDC" NCHAR(40),
"ABMCU" NCHAR(12),
"ABSIC" NCHAR(10),
"ABLNGP" NCHAR(2),
"ABAT1" NCHAR(3),
"ABCM" NCHAR(2),
"ABTAXC" NCHAR(1),
"ABAT2" NCHAR(1),

…

"ABACTIN" NCHAR(1),
"ABREVRNG" NCHAR(5),
"ABSYNCS" NUMBER,
"ABPERRS" NUMBER,
"ABCAAD" NUMBER )
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255
 NOCOMPRESS LOGGING
  STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
  BUFFER_POOL DEFAULT )
  TABLESPACE "PRODDTAT"
  PARALLEL 8 ;

select to_char(sysdate,'HH24:MI:SS') from dual;
select to_char(sysdate,'YYY MM DD') from dual;


  INSERT INTO PRODDTA.F0101 SELECT * FROM PRODDTA.F0101_NONUNI ;
commit ;

select to_char(sysdate,'HH24:MI:SS') from dual;
select to_char(sysdate,'YYY MM DD') from dual;

spool off;
quit;

There has been a massive change in mobile app development over the last couple of tools releases. The latest modifications are fantastic.

Start your learning journey here:

https://support.oracle.com/epmos/faces/DocContentDisplay?_afrLoop=925295454654543&id=1932603.1&_afrWindowMode=0&_adf.ctrl-state=z3b4xcg6i_501

You’ll need support.oracle.com credentials.  The documentation is awesome and so are the examples.  They are making it very easy to use tested and existing application functionality in the mobile space.

http://docs.oracle.com/cd/E24705_01/doc.91/e56635/img/jde_e1_mobile_arch.gif

This is probably some sort of infringement, but it’s for the greater good.  You can see now that there is an AIS server (J2EE container) which is the gateway between the mobile device and the application data and logic via the “FormServiceRequest” method available from the native E1 web installation.  Cool!

So essentially you are turning an E1 form into a class, with standard CRUD type functionality.

http://docs.oracle.com/cd/E24705_01/doc.91/e56635/img/json_in_and_out.png

So it seems to be that this is a great way of using what has been written in JD Edwards and using your existing development skills to be able to fairly quickly enable mobile interactivity with JD Edwards.

You’ll need to skill up with JDeveloper – but that’s easy enough.  A Number of helper classes have been provided for you to assist this journey.

Of course error handling is going to be fun (it always is).  Media objects are going to be fun, but they always are aren't they?

This is pretty simple, getting the script / server to return some output to the output of SQLPlus.  I saw a lot of articles on this, but non complete, i.e. – did not have the SET SERVEROUTPUT line, therefore, nothing was being printed.

I’m doing unicode conversions manually and need to drop the temp files once I’ve confirmed that all of the data rows have been converted.  Therefore I get a couple of counts (from the nonuni and singlebyte) versions of the tables.  If the counts are the same, then I drop the “_NONUNI” – simple!

This script is just for reference and does not have the drop command.

SET SERVEROUTPUT ON FORMAT WORD_WRAPPED
declare
unicode_count integer;
singlebyte_count integer;
cursor unicode_cursor is select count(1) from TESTDTA.F0101;
cursor singlebyte_cursor is select count(1) from TESTDTA.F0101_NONUNI;

begin

  open unicode_cursor;
  fetch unicode_cursor into unicode_count;
  open singlebyte_cursor;
  fetch singlebyte_cursor into singlebyte_count;
  if unicode_count <> singlebyte_count then
    dbms_output.put_line('cannot drop table, transation not complete');
  else
    dbms_output.put_line('Drop the temp table, all good');
  end if;
  close unicode_cursor;
  close singlebyte_cursor;

end;
/
quit ;
/

I need to drop tables that have been verified as unicode converted, I’m no expert at PL/SQL, so I decided to make a function that was generic.

What I decide to do is if the NONUNI = normal row count, my assumption is that everything has worked like a peach, this is going to help me do this.

Note that the drop has not been coded, because I’m a little worried…

CREATE OR REPLACE FUNCTION test_unicode_conversion (singlebyte_tablename in VARCHAR, unicode_tablename IN VARCHAR)
RETURN VARCHAR
IS
unicode_count integer;
singlebyte_count integer;
 unicode_cursor sys_refcursor; 
 singlebyte_cursor  sys_refcursor;
BEGIN
  open unicode_cursor for 'select count(1) from ' || unicode_tablename  ;
  fetch unicode_cursor into unicode_count;
  open singlebyte_cursor for 'select count(1) from ' || singlebyte_tablename ;
  fetch singlebyte_cursor into singlebyte_count;
  if unicode_count <> singlebyte_count then
    dbms_output.put_line('cannot drop table, transation not complete');
    return 'Row counts do not match for' || unicode_tablename || ' and ' || singlebyte_tablename ;
  else
    dbms_output.put_line('Drop the temp table, all good');
    --drop table PRODDTA.F55B30 ;
    return 'Row counts MATCH for' || unicode_tablename || ' and ' || singlebyte_tablename ;
  end if;
  close unicode_cursor;
  close singlebyte_cursor;
END ;

select jde.test_unicode_conversion('TESTDTA.F0101', 'TESTDTA.F0101') from dual ;

And calling this ripper:

select jde.test_unicode_conversion('TESTDTA.F0101', 'TESTDTA.F0101') from dual ;

Which returns:

Row counts MATCH forTESTDTA.F0101 and TESTDTA.F0101

That is pretty cool.  Note that there are a number of specifics (syntax is a killer!)

The important parts about this script is the pragma autonomous_transaction and also the use of execute immediate for the drop table.  This functional also has an additional parameter for determining whether to drop the table or not – allowing some prudent testing.

Remember the first parameter is the singlebyte tablename and the second is the unicode_tablename.

If the third parameter is a Y, and you have permission, the table will be dropped.

Another cool feature is that this information is returned to the query (and therefore spooled).

CREATE OR REPLACE FUNCTION test_unicode_conversion (singlebyte_tablename in VARCHAR, unicode_tablename IN VARCHAR, dropTable IN CHAR)
RETURN VARCHAR
IS
unicode_count integer;
singlebyte_count integer;
 unicode_cursor sys_refcursor; 
 singlebyte_cursor  sys_refcursor;
 pragma autonomous_transaction;
BEGIN
   dbms_output.put_line('Starting...');
  open unicode_cursor for 'select count(1) from ' || unicode_tablename  ;
  fetch unicode_cursor into unicode_count;
  open singlebyte_cursor for 'select count(1) from ' || singlebyte_tablename ;
  fetch singlebyte_cursor into singlebyte_count;
  if unicode_count <> singlebyte_count then
    dbms_output.put_line('cannot drop table, transation not complete');
    return 'Row counts do not match for' || unicode_tablename || ' and ' || singlebyte_tablename ;
  else
    dbms_output.put_line('Drop the temp table, all good');
    if droptable = 'Y' then
      execute immediate 'drop table ' || singlebyte_tablename ;
      return 'Row counts MATCH for ' || unicode_tablename || ' and DROPPED ' || singlebyte_tablename ;
    end if;
    return 'Row counts MATCH for' || unicode_tablename || ' and ' || singlebyte_tablename ;
  end if;
  close unicode_cursor;
  close singlebyte_cursor;
END ;

Called with

select jde.test_unicode_conversion('TESTDTA.F4009_NONUNI', 'TESTDTA.F4009', 'N') from dual ;

This is a more generic problem that changes that you are making to files in your weblogic user_project directory are not being reflected in the clients browser.  A simple change to a jsp for your EA is not being shown. 

You’ve restarted the app on a number of occasions and you are thinking that you are going mad.  Are you changing the wrong files?  What is going on.

c$\Oracle\Middleware\Oracle_Home\user_projects\domains\base_domain\servers\JDEDV_8080

Look for the directory equivilent above on your webs erver and delete the tmp and cache dirs while the WLS instance is down.  You’ll notice that the size of the tmp dir is about the same size as your EA – give or take.

Once this is done, start the application and your changes are going to be there

Is is just me, or does this happen every time I update SM…  It goes bad…

I just chose to update my SM to 9.1.5.2 and of course, when I try to login, I get the above – GREAT!

Resource not found on this server.

This is where you need to trusty old recovery commands:

How to troubleshoot an unsuccessful Management Console update?

To troubleshoot an unsuccessful Management Console update:

1. Stop the Management Console service.
2. Start the Management Console service.
3. Open a command prompt and navigate to this directory:
      \jde_home\targets\home\_staging
       where jde_home is the install path of the Management Console.
4. Invoke the redeployManagementConsoleredeployManagementConsole.cmd recovery script.

The script takes one parameter, which is the password used to initially install the Management Console.
For example: redeployManagementConsoleredeployManagementConsole.cmd password

Upon successful execution of this script you can log on to the Management Console application.

And viola:

15/02/24 21:56:25 Notification ==>Initialize ManagementConsole_WAR ends...

15/02/24 21:56:25 Notification ==>Started application : ManagementConsole

15/02/24 21:56:25 Notification ==>Binding web application(s) to site default-web
-site begins...

15/02/24 21:56:25 Notification ==>Binding ManagementConsole_WAR web-module for a
pplication ManagementConsole to site default-web-site under context root /manage

15/02/24 21:56:28 Notification ==>Initializing Servlet: com.jdedwards.mgmt.web.C
onsoleInit for web application ManagementConsole_WAR

15/02/24 21:56:38 Notification ==>Initializing Servlet: oracle.cabo.servlet.UIXS
ervlet for web application ManagementConsole_WAR

15/02/24 21:56:39 Notification ==>Binding web application(s) to site default-web
-site ends...

15/02/24 21:56:39 Notification ==>Application Deployer for ManagementConsole COM
PLETES. Operation time: 50338 msecs

We are away again…

I have my media objects in a flat file structure on the deployment server and need to do some analysis in oracle on my F00165.  I really need to see what physical files still exist and then populate F00165 with the new file locations (yes we are moving).  This is a continuation of a series of emails about media objects and how they can go wrong.

This particular situation is at a client that has about 20 million records in F00165 and reference to over 5 000 000 physical files.  You may or may not know that trying to restore this amount of physical files to a single directory in windoze is going to kill the system.  The file allocations and internal referencing for a folder seems to crap out at about 100 000 files, and the copy then takes longer than me doing it on an abacus and arranging the bits and bytes with a magnet.

So I’ve created a oracle table with the command:

create table proddta.f00165 (mofilename varchar(256)) ;

Then ran the script

Set FSO = CreateObject("Scripting.FileSystemObject")

Dim Oracon
set oraccon = wscript.createobject("ADODB.Connection")
Dim recset 
set recset = wscript.createobject("ADODB.Recordset")
set Insertresults = wscript.createobject("ADODB.Recordset")
set f98moquerecset = wscript.createobject("ADODB.Recordset")
Dim cmdInsert 
set cmd = wscript.createobject("ADODB.Command")
set cmdInsert = wscript.createobject("ADODB.Command")
set cmdf98moque = wscript.createobject("ADODB.Command")
Set Oracon = wscript.CreateObject("ADODB.Connection")

Oracon.ConnectionString = "DSN=pjde;"& _
"User ID=jde;"& _
"Password=jde;"

Oracon.Open
Set cmdInsert.ActiveConnection = Oracon

ShowSubfolders FSO.GetFolder("Z:\MediaObjectsData\HTMLUploads"), 3 

Sub ShowSubFolders(Folder, Depth)
    If Depth > 0 then
        For Each Subfolder in Folder.SubFolders
            if(instr(subfolder,"RECYCLE")=0) then
'Wscript.Echo Subfolder.Path
              ShowSubFolders Subfolder, Depth -1 
		Set colFiles = subFolder.Files
		For Each file In colFiles
    		  absPath = FSO.GetAbsolutePathName(file)
'wscript.echo absPath
    		  cmdInsert.CommandText = "INSERT INTO PRODDTA.F00165SRM VALUES ('"& absPath & "')"
    		  Set recset = cmdInsert.Execute
		Next
            end if
        Next
    End if
End Sub

Note that 

Note that "Z:\MediaObjectsData\HTMLUploads" is the root of the copy of the media objects.  We had to use a series of sub folders because of the windoze problems with performance.

So then I had my oracle table with a FULL path to all of the media objects.

I followed the above with some SQL against the F00165 to update the location of many of the physical file references that are wrong.

update proddta.f00165
set gdgtfilenm = (select replace(mofilename, 'Z:\','\\jdedatastore\') 
    from  proddta.f00165srm 
    where substr(mofilename,instr(mofilename,'\',-1,1)+1, length(trim(mofilename))-(instr(mofilename,'\',-1,1))) = substr(gdgtfilenm,instr(gdgtfilenm,'\',-1,1)+1, length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1))) )
where  gdgtmotype = '5'
and gdgtfilenm like '\\%'
and length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1)) > 1
and exists
(select 1 from
proddta.f00165srm
where substr(mofilename,instr(mofilename,'\',-1,1)+1, length(trim(mofilename))-(instr(mofilename,'\',-1,1))) = substr(gdgtfilenm,instr(gdgtfilenm,'\',-1,1)+1, length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1))) 
and length(trim(mofilename))-(instr(mofilename,'\',-1,1)) > 1);

Have you ever wanted to update a table based upon a join.  For example, wanted to update F00165 based upon a join of F00165 and another table?  This is really hard to talk about generically, so lets get specific.

I have a table F00165 with all my media objects.  I have another table F00165SRM that has the location of all of the NEW media objects that exist.  So now I want to update F00165 and set the gdgtfilenm to equal the new value.

Oracle does not support joins in update statements.  I was trying to do a complex where exists type query, but it was taking too long.

I can query the results easy enough, but hard to turn this into an update statement:

select replace(mofilename, 'Z:\','\\jdedatastore\') , substr(mofilename,instr(mofilename,'\',-1,1)+1, length(trim(mofilename))-(instr(mofilename,'\',-1,1)))
    from  proddta.f00165srm, proddta.f00165
    where instr(mofilename,'\',-1,1) > 1
    and instr(gdgtfilenm,'\',-1,1) > 1
    and  length(trim(mofilename))-(instr(mofilename,'\',-1,1)) > 5
    and length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1)) > 5
    and substr(mofilename,instr(mofilename,'\',-1,1)+1, length(trim(mofilename))-(instr(mofilename,'\',-1,1))) = substr(gdgtfilenm,instr(gdgtfilenm,'\',-1,1)+1, length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1)))
    and length(trim(gdgtfilenm)) > 50;

So I stumble on the MERGE statement, and this makes me very happy!

  MERGE
  INTO    proddta.f00165 t3
  USING   (
          SELECT  t1.rowid AS rid, replace(t2.mofilename, 'Z:\','\\jdedatastore\') as newMOLocation
          FROM    proddta.f00165 t1
          JOIN    proddta.f00165srm t2
          ON      substr(mofilename,instr(mofilename,'\',-1,1)+1, length(trim(mofilename))-(instr(mofilename,'\',-1,1))) = substr(gdgtfilenm,instr(gdgtfilenm,'\',-1,1)+1, length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1)))
          WHERE   instr(mofilename,'\',-1,1) > 1
            and instr(gdgtfilenm,'\',-1,1) > 1
            and  length(trim(mofilename))-(instr(mofilename,'\',-1,1)) > 5
            and length(trim(gdgtfilenm))-(instr(gdgtfilenm,'\',-1,1)) > 5       
            and  gdgtmotype = '5')      
  ON      (t3.rowid = rid)
  WHEN MATCHED THEN
  UPDATE
  SET     t3.gdgtfilenm = newMOLocation;

As you can see from the above this seems to be a nice and simple way of doing the update of a single table with the join of multiple tables.  The other nice thing is that you can also do WHEN NOT MATCHED and apply some other logic.  See oracle doco here http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_9016.htm#SQLRF01606 

I do however get an ORA-04036 when running it – so perhaps there is a little fine tuning required.  Nice new 12C feature that it hitting me.