Pre- and post work of an SAP System refresh
Veröffentlicht von Shortcut IT in Sc4SAP · 26 Juni 2023
Tags: SAP automation;SAP Systemkopie;SAP system copy;SAP system refresh;R3trans;PCA
Tags: SAP automation;SAP Systemkopie;SAP system copy;SAP system refresh;R3trans;PCA
It is a recurring task for the SAP Basis: the data of the quality and test systems become outdated, are thus less and less useful and must therefore be periodically updated with data from the productive system. For the SAP Basis, this is a high effort, and also under time pressure. After all, the systems should be available again quickly. With "Shortcut for SAP systems", most of the pre- and post-processing can be automated.
The (simplified) sequence in a system refresh is this:
Step 1: Export the data to be kept (mostly system-specific configuration data) on the target system
Step 3: Cleanup of some data in the target systemStep 2: Database copy from source to target systemStep 4: Import data (exported in step 1) back into the target system
In this article we deal with steps 1, 3 and 4. The database copy (step 2) is not considered here.
In many how-to's that you can find in the web, you are advised to create screenshots and then configure the system based on the created screenshots. This is sooooo outdated! It requires a lot of manual work both in preparation and in post-processing, with all the resulting disadvantages, e.g.:
- If something is forgotten during preparation - e.g. a screenshot - it may not be possible to correct it during the post processing.
- Manual activities are prone to errors.
- The processing times for manual activities are longer than for restoring saved data.
- The manual activities may have to be carried out at "inconvenient" working times, which may again increase the susceptibility to errors.
Therefore, the approach with manual activities should be left behind as far as possible and as much of the pre- and post-processing as possible should be done by backup / restore.
In this article I would like to point to the "PCA" tool and how we can use it. "PCA" stands for "Post Copy Automation" and was developed by SAP for supporting system copies / refreshes. However, you need a "SAP Landscape Virtualization Management (LVM), enterprise edition license" (see SAP note 1589175) to use it. If this is given, you may follow the installation guide and use the Task Manager (transaction STC01) for doing the pre work and the post work.
If you do not have the required license, you will see in this article, how information from the PCA tool can be used and a lot of pre- and post work can be done using "Shortcut for SAP systems".
So let's see what the PCA tool can do for us. Hmm... I already wrote that we need a licence for this, so what else can we expect here? With the PCA tool we can get the information of the tables to be backed up (and restored after the system refresh). Even if the license may not be there and the configuration of the PCA tool requires additional effort, we can use it for getting the information about the relevant tables.
There is a report SCTC_LIST_TABLES available in every(?) SAP NetWeaver system, that is useful for showing the relevant tables.
Ok, there are some values to be given as input: a choice between "Refresh" and "Data Cleanup" and an obligatory "Component". Unfortunately there is no F4 list available for the "Component". However, to shorten this: the execution of this report causes some problems. We do not know the "Component" up to now, and even if we would (try "RFC" for example), we would be faced with an error message:
Despite the program is not running, we can still use it for getting the information. A look into the source code shows that the information about the relevant components is hard-coded in function module SCTC_GET_ALL_COMPONENTS and from there the information about the assigned tables is in Include SCTC_SC_INCL_COMPONENTS.
You can use this program, which is a modified and extended copy of it. You can find it also supplied with our product in the folder 'ABAPs' (file 'ShowPCATables.txt'). Just create it in the customer name space of the development system. Most of the PCA tables are hard-coded in ABAP source codes, some are determined from the TADIR table (R3TR TABL from (also hard-coded) packages).
Let's have a look at the selection screen of the program:
First part with "Refresh" and "Data Cleanup" is similar to the SAP program. What is this "Refresh" and "Data Cleanup" about?
- The "Refresh" option belongs to tables that are to be exported in front of the system refresh and to be imported after the system refresh.
- The "Data Cleanup" option belongs to tables that are to be emptied after the system refresh. For example, the tables containing data for spool requests, asynchronous updates and background RFC processing are part of the "Data Cleanup".
Ok, for now, let's ignore the other input fields on the selection screen and look at the tables. We execute the program with the "Refresh" option - to see the tables that should be backed up before (step 1) and restored after the system refresh (step 4):
You will find in the list all components and their tables, that are handled in the PCA tool: ALE stuff, Batch Input, SAP Office, printer, operation modes etc. - lots of stuff that possibly you are already aware of and dealed with it more or less cumbersome by taking screenshots before and repeating the configuration tasks after the system refresh. And maybe there is some stuff you were not aware of in the past.
In addition to the original SAP program SCTC_LIST_TABLES you will also find some information about the table size, the amount of rows, whether the table is client dependent or not and a button enabling you to jump to the Data Dictionary information for the tables (SE11).
Ok, so now we have the information about the components and their tables which are worth to be managed in the context of a system refresh. How can we use this information now?
Have a look at the "XML file" button at the top of the list. Such a small button, but it can save us soooo much work.
The generated XML file is for using it with the "Shortcut for SAP systems" command line tool. The command line tool enables you to execute nearly all functions of "Shortcut for SAP systems" outside of the GUI application, executable on OS level.
Let's try it. We are asked about a location for the file:
After saving the file, a popup with some information comes up:
There is a difference of 'Total tables' to 'Distinct tables'. This is because some tables are listed in more than one component (e.g. the components ARCHIVE, ARCHIVE_ADK and ARCHIVE_CUST have some intersections regarding their tables, same is for components CRM, CRMINERP, PI_BASIS_CRM and some more).
Let's have a look into the XML file. There are 2 of them: one file was created for exporting the data - to be done before the system refresh takes place - and one file for the import - to be done after the system refresh, to recover the saved data.
And what's inside? In the "Export" file we see lots of tasks, one for each component and all their tables from the list:
So this is the export of the data, and accordingly in the "Import" file we see the corresponding import of the data.
For exporting/importing "R3trans" is used. "R3trans" is an SAP utility and is mainly used in the Transport Management System (TMS) environment. R3trans can be used to read, save and import table data from the SAP system - and even independently of the database. Also the PCA tool uses R3trans.
In the file we find some information from the selection screen, e.g. the connection data ("+++ Enter the connection here +++"). So let's go one step back and have a more detailed look on the selection screen.
In the first elements of the selection screen you find the differentiation between "Refresh" and "Cleanup", already mentioned earlier in this article. There is also a flag "Without components NRIV and SNRO". For these components all records of the NRIV table would be considered.
Our aim is to get recent data of the productive system via system refresh - including all the business documents, where lots of them have a document number, received from the number range method based on the NRIV table. If we export the whole NRIV table - the last given number for each number range object is part of it - and restore it after the system refresh, we will run into problems when it is about creating new documents. Assuming that on our Quality system currently there are less documents than on the Productive system, the last given number in the NRIV table is already occupied.
In a nutshell: leave the flag set and with this leave out the components SNRO and NRIV. For the XML files the area below is of interest: you can create XML files for backing up system specific data or for using them in the context of a system refresh (2), which we do here. In the next input field (3) we enter the connection to our target system. In field (4) we specify a directory for the data files (preferably a dedicated directory for this purpose). This directory has to exist resp. created and of course it has to be accessible from the SAP system. As it is here about a system refresh (where usually "only" the database files are copied from the source to the target system), it seems to be a good idea to use a directory directly on the SAP server.
The fields below deal with performance aspects of the export / import. In this article you will find more information.
Now let's execute the program and generate the XML files again.
Again we have an XML file for export and another one for import. Both now contain the given connection. Of course we could have achieved the same result by just editing the line with the <DefaultConnection> tag in the XML files - it is "only" XML and can be modified with a text editor.
For our system refresh we miss the "Cleanup" part up to now, so we go again one step back and generate the XML file for the cleanup.
The amount of components and tables is smaller than for export / import.
So now we have these 3 XML files:
- QX1_SystemRefresh_Export.xml
- QX1_SystemRefresh_Remove.xml
- QX1_SystemRefresh_Import.xml
I move the files to a dedicated directory "E:\SAP\scripts". Now let's do the export of the data and call the command line tool for processing the XML file for export.
During the execution of the tasks you can have a look into the specified target directory and see the files being created:
After few minutes (which is of course depending on the amount of data in your system) the execution finished.
The XML file is now enriched by detailed information for each task. And at the end of the XML file we find a summary:
Lots of tasks finished with returncode 4 - a warning. Having again a look into AL11 we find for every R3trans file also a log file. Let's have a look into one of them:
R3trans works in general with a "control file" (containing the R3trans command, the specification of the client etc.) and - optionally - with a "command file", which is in fact a transport request. The tables to be processed can be specified in the control file and/or in the command file. For enabling R3trans to use parallelization (performance often becomes an issue for system refreshes), the tables are to be specified in a command file. The warning here results out of the use of a command file. When using the PCA tool, you will find the same warning. Not for every component a command file is used, therefore some tasks ended with RC 0.
Ok, it worked. With this we have done the export of the data in front of the system refresh.
Now let's make a jump to the time when the systems database is overwritten with the one from the source system. As you know some data has to be adjusted - lots of settings like SAP Connect, the server groups, CCMS configuration, TMS configuration, logical paths and files, the SAP license, logon groups, operation modes etc. etc. etc.
We are going to do this by restoring the saved data. All of the mentioned topics are covered in the tables we saved in front of the system refresh - so by restoring these data we will have the same status in the overwritten system as before.
Start with the "Cleanup" task. We already have our XML file for the command line tool, so it is just about executing the command line tool with the file as argument.
Again there are tasks ended with return code 4 (a warning). The log files show the reason of it:
Here a warning was given because all tables were already empty. R3trans had nothing to do, so it quitted with a warning.
There were some more return codes 4, and all of them had the same reason.
If you have a look into the SAP system, you will find some impacts of the data cleanup. For example, in STMS you will see the message "TMS not configured for this SAP System", the logon groups in SMLG and RZ12 are gone, SM13 and SM35 are empty, same with ST22.
The Data cleanup part took ~1 minute only on the system. As it is about deletion of data and the amount of tables is smaller than for the previously done 'Export' part, also the needed time is less.
Now let's do the last step (and the most exciting of all): importing the saved data again. As we already have our XML file, it is again just about executing the command line tool.
Again there are tasks with RC 4, so have a look into the log files.
"source system == target system." Yes, that is exactly what we did - export and import on the same system. We can ignore these warnings, they are the same for every task.
The import tasks needed significantly more time than the export tasks: ~27 minutes on my small system. This is because updating the tables in general needs more time than just reading the data. Beside others indexes are to be considered, rollback segments are to be managed by the database etc.
The log files look a little bit different from the 'export' ones, pointing to the fact that it is here about update / insertion instead of simple reading:
Let's have a look into the system now. The TMS looks fine again, also the batch input sessions and the SAP Connect settings in transaction SCOT look like before the system refresh.
All the configuration settings are the same as before the system refresh. This is a huuuuge gain of saved time in comparison to do the pre and post processing manually, using screenshots and do lots of configuration stuff to get the status of the system back as it was before the system refresh. And, if you up to now used a documentation with lots of manual activities, I guess dealing with system specific data using the above described method is much more complete than doing this using your documentation, isn't it?
Once you have finished the adjustment of the XML files for pre-work (Export) and post-work (Data cleanup and Import), you can use it easily to do the pre- and post-processing as often as you like - so, if there is the demand to do the system refresh every month or even every week, there is no need to be afraid of the previously huge amount of work anymore! At the end it is about 3 calls of the command line tool.
As this is working on OS level, you can also easily schedule these tasks or integrate it in already existing automation solutions. Have your weekends been messed up for this in the past? That can now come to an end.
Finally I would like to give some hints on this topic:
- Of course it is on you to decide which components and tables are to be considered. If you do not want to deal with the full scope of data, you can filter the list of the program Z_SHOW_PCA_TABLES and / or de-select tables. Possibly you want to get the batch input sessions from the source system, then just leave out the BATCH_INPUT component for Export, Data Cleanup and Import. Just take the list of the program, think about it, discuss and agree with your colleagues and/or the user department, which components are to be considered and which not.
Filter and selection in the list of Z_SHOW_PCA_TABLES will be considered when generating the XML file - means, components and tables that are filtered out or de-selected will not be considered in the generation of the XML file. Certainly, you can also modify the XML file in a text editor and take out some tasks or add others. - In the 'xml' folder of "Shortcut for SAP systems" you will find XML examples for pre- and post work of a system refresh. Have a look into it, they show some additions: in the pre-work locking and kicking out of users is integrated as well as suspending batch jobs in front of data saving. And in addition to the tables shown by the PCA tool, also the ALV layouts are considered. Consequently this is also implemented in the post-processing example.So you can inform the users about the upcoming system refresh and schedule the command line tool with the pre-processing XML file e.g. for Friday evening - no need to do it on your own and stay in the office. Turn this into an automatism.
- You can significantly speed up the process (especially the import, which consumes the most time) by using parallelization. Find detailed information how to do so in this article.
- By default also the users are considered in the PCA tables. If you do not want this, leave out the USERS component. For some reasons it makes sense to receive all users and their role assignments from the source system, just like you receive material master data, FI documents etc. from the source system - finally, it is the aim of a system refresh to get data close to the productive environment.
However, the system users have to be the same as before the system refresh, otherwise interfaces would not work anymore. And possibly project teams etc. are also to be kept.
You also have the possibility to keep all users in the target system (by exporting / importing the USERS component) and copy additional users from the source system to the target system with the "Copy user" function.
Have a look at these blog articles regarding this topic:
https://www.shortcut-it.com/blog/index.php?how-to-assign-new-passwords-to-lots-of-usersCopying users and setting passwords can also be done in the XML files, so you no longer have to do this manually either.
Es gibt noch keine Rezension.