The power of knowledge warehousing

Object class violation when creating group in LDAP using WebSphere Portal

2010-04-29T16:51:00.002+07:00

Technote (troubleshooting)

Problem(Abstract)

When attempting to create a group using IBM® WebSphere® Portal, you receive an "EJPSG0015E: Data Backend Problem" error. The details show that this is caused by the LDAP server returning an error code 65 (object class violation).

Symptom

SystemOut.log will contain:

... WSMM Message E com.ibm.ws.wmm.ldap.LdapRepositoryImpl void create(String DN, Attributes attributes) The following Naming Exception occured during processing: "javax.naming.directory.SchemaViolationException: [LDAP: error code 65 - Object Class Violation]; remaining name 'cn=testgroupname,o=ibm'; resolved object com.sun.jndi.ldap.LdapCtx@28da7087".
... CreateGroupCo E com.ibm.wps.command.puma.CreateGroupCommand AbstractCommand.throwCommandFailedException EJPEB0003E: CreateGroupCommand.execute: An exception occurred.
com.ibm.wps.util.DataBackendException: EJPSG0015E: Data Backend Problem
...
Caused by: com.ibm.websphere.wmm.exception.WMMSystemException: The following Naming Exception occured during processing: "javax.naming.directory.SchemaViolationException: [LDAP: error code 65 - Object Class Violation]; remaining name 'cn=testgroupname,o=ibm'; resolved object com.sun.jndi.ldap.LdapCtx@28da7087". at com.ibm.ws.wmm.ldap.LdapRepositoryImpl.create(LdapRepositoryImpl.java:107)

Resolving the problem

Refer to the 'LDAP: error code ...' technote in the related information below which explains that the LDAP server generates the error code 65. Consult with your LDAP administrator to troubleshoot the failure from an LDAP perspective.
You may have to update the WebSphere Member Manager (WMM) configuration to resolve the problem. For example, this error can occur when attempting to create a group in IBM Tivoli Directory Server (ITDS) using the objectClass accessGroup. Consider the case where wmm.xml contains:

<supportedLdapEntryType name="Group" rdnAttrTypes="cn" objectClassesForRead="accessGroup" objectClassesForWrite="accessGroup" searchBases="o=ibm"/>

and wmmLDAPServerAttributes.xml contains:

<attributeMap wmmAttributeName="groupMember" pluginAttributeName="uniqueMember" applicableMemberTypes="Group" dataType="String" valueLength="1024" multiValued="true" defaultValue="uid=dummy"/>
If you enable the traces recommended in the WebSphere Member Manager MustGather and attempt to create a group in WebSphere Portal, you will see the following in trace.log:
.WMM Trace Log > com.ibm.ws.wmm.ldap.LdapConnectionImpl DirContext createSubcontext(Name name, Attributes attrs) Entry cn=testgroupname,o=ibm {objectclass=objectClass: accessgroup, uniquemember=uniquemember: uid=dummy, cn=cn: testgroupname}
...
WSMM Message E com.ibm.ws.wmm.ldap.LdapRepositoryImpl void create(String DN, Attributes attributes) The following Naming Exception occured during processing: "javax.naming.directory.SchemaViolationException: [LDAP: error code 65 - Object Class Violation]; remaining name 'cn=testgroupname,o=ibm'; resolved object com.sun.jndi.ldap.LdapCtx@1faad592"

This failure occurs because the definition of the accessGroup object class does not include uid and uniquemember attributes:

accessgroup :
{
1.3.18.0.2.6.75
NAME 'accessGroup'
DESC 'A group that is used for access control.'
SUP top STRUCTURAL
MUST cn
MAY (businessCategory $ description $ member $ o $ ou $ owner $ seeAlso)
}
Therefore, to correct the problem you have two options. If your LDAP server is configured such that a defaultValue is required for member (see "Mapping LDAP Attributes" link below), then you must update wmmLDAPServerAttributes.xml replacing the entry for wmmAttributeName="groupMember" with (changing pluginAttributeName and defaultValue):

<attributeMap wmmAttributeName="groupMember" pluginAttributeName="member" applicableMemberTypes="Group" dataType="String" valueLength="1024" multiValued="true" defaultValue="cn=dummy"/>
If your LDAP server does not require a defaultValue, you could use the following instead:

<attributeMap wmmAttributeName="groupMember" pluginAttributeName="member" applicableMemberTypes="Group" dataType="String" valueLength="1024" multiValued="true"/>
Use the check-out/check-in procedure to update the WMM configuration if running in a cluster. Restart WebSphere Portal to read the updated WMM configuration.

Related information

'LDAP: error code ...' messages
WMM MustGather
Mapping LDAP Attributes

Detailed steps for mapping LDAP attributes in WebSphere Portal 6.1

2010-04-07T16:29:00.003+07:00

Technote (FAQ)

Question

How can you map attributes in IBM WebSphere Portal 6.1? It appears that there are unmapped attributes by default when initially configuring WebSphere Portal to an LDAP. WebSphere Portal Information Center does not provide detailed steps to explain the process.

Answer

The following step-by-step instructions address attribute mapping in WebSphere Portal and should be considered as supplemental information to the WebSphere Portal Information Center page, Mapping attributes. The instructions in this technote use a stand-alone security setup on Linux with Tivoli Directory Server as the LDAP. However, they can be adjusted to address other LDAPs or operating systems, and can be modified for federated repository setups by using the appropriate properties as outlined on the Information Center page.

------------------

1. Run "ldapsearch" from the Linux console to determine available attributes in LDAP. If you do not have an ldapsearch tool, refer to the related information at the bottom of this technote for a link to a command-line tool.

Command:

Results:

Command:

Results:

------------------

2. Run ./ConfigEngine.sh wp-query-attribute-config to produce output similar to the following in the <wp_profile>/ConfigEngine/log/Configtrace.log:

and then use the above information to modify the wkplc.properties file:

------------------

Additional output in the ConfigTrace.log:

[wplc-validate-ldap-attribute-config] The following attributes are flagged as required in LDAP but not in Portal - You should flag them as required in Portal, too:
[wplc-validate-ldap-attribute-config] [sn, cn]

Modifiy the wkplc.properties file accordingly:

user.attributes.required=sn,cn
-----------------

Additional output in the ConfigTrace.log:

[wplc-validate-ldap-attribute-config] FYI: The following attributes have a diffenrent type in Portal and in LDAP - No action is required:
[wplc-validate-ldap-attribute-config] jpegPhoto: Base64Binary <> 1.3.6.1.4.1.1466.115.121.1.5
[wplc-validate-ldap-attribute-config] password: Base64Binary <> 1.3.6.1.4.1.1466.115.121.1.5
[wplc-validate-ldap-attribute-config] seeAlso: String <> 1.3.6.1.4.1.1466.115.121.1.12

No modifications are required in the wkplc.properties file.

-------------------

Additional output in the ConfigTrace.log:

[wplc-validate-ldap-attribute-config] Possible problems for Group:

[wplc-validate-ldap-attribute-config] The following attribues are defined in Portal but not in LDAP - You should either flag them as unsupported or define an attribute mapping:[wplc-validate-ldap-attribute-config] [modifyTimestamp, groups, members, identifier, displayName, parent, children, entitlementInfo, partyRoles, viewIdentifiers, createTimestamp]

Modify the wkplc.properties file accordingly:

-----------------

Additional output in the ConfigTrace.log:

[wplc-validate-ldap-attribute-config] The following attributes are flagged as required in LDAP but not in Portal - You should flag them as required in Portal, too:
[wplc-validate-ldap-attribute-config] []

No modifications are required in the wkplc.properties file.

-----------------

NOTE: Leave the following line unchanged in the wkplc.properties file:

standalone.ldap.attributes.mapping.entityTypes=PersonAccount,Group
-----------------

3. Run ./ConfigEngine.sh wp-update-standalone-ldap-attribute-config and ensure you receive a BUILD SUCCESSFUL message.

4. Run ./ConfigEngine.sh wp-query-attribute-config and ensure you receive a BUILD SUCCESSFUL message.

5. Once all required attributes are mapped, run the following tasks to propagate the security changes in a standalone (non-clustered) environment:

admin_userid

admin_password

wp_profile_root

WebSphere_Portal

admin_userid

admin_password

wp_profile_root

WebSphere_Portal

wp_profile_root

http://www.ibm.com/support/docview.wss?rs=688&ca=portall2&uid=swg21365605

Related information

Mapping Attributes
Using ldapsearch to debug LDAP configuration problems
Exception related to attributes

Messages In Console: Oracle(9581): Floating-Point Assist Fault At Ip [ID 279456.1]

2010-04-02T10:51:00.002+07:00



		Modified 27-JAN-2010 Type PROBLEM Status PUBLISHED

In this Document
Symptoms
Cause
Solution
References

Applies to:

Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 10.1.0.2
Red Hat Enterprise Linux Advanced Server Itanium
UnitedLinux Itanium
Oracle Server Enterprise Edition - Version: 10.1.0.2 to 10.1.0.2
Checked for relevance on 25-Feb-2009

This note is applicable on Itanium Architecture only

Symptoms

In the server console appears messages like

oracle(9581): floating-point assist fault at ip 40000000068c22c2
...
oracle(13763): floating-point assist fault at ip 40000000072b8081

Cause

The messages are not errors,
just information/warning that the operation being performed by the processor needed a software assist.

As reported on
http://h21007.www2.hp.com/portal/site/dspp/menuitem.863c3e4cbcdc3f3515b49c108973a801?ciid=62080055abe021100055abe02110275d6e10RCRD

We reproduce here part of this article titled What's up with those "floating-point assist fault" messages? - Linux on Itanium

When working at the Linux console or looking through the kernel's log file in /var/log/messages,
you may encounter messages of the form:

test-fpsr(1416): floating-point assist fault at ip 40000000000005d2

This message means that program "test-fpsr" with process id 1416 performed a floating-point
operation that required software assistance.
On Itanium, this usually happens when operating on IEEE denormals
(numbers that cannot be represented in normalized form).

Again, just like for unaligned accesses, these operations are emulated in the kernel,
so there is nothing to worry from a correctness point of view. However,
emulating these operations takes time and can slow down a program significantly.
Since most programs don't need full IEEE-compliance, Itanium Linux
provides a function to turn on "flush-to-zero" mode in the processor,
which avoids generating IEEE denormals as a result of ordinary floating-point calculations.
This mode can be turned on using the following code fragment:

#include
:
fesetenv (FE_NONIEEE_ENV);
:

Alternatively, compiler option -ffast-math can be used when linking
the final program to achieve the same effect. For example:

gcc -ffast-math test.o -o test

would build program "test" with flush-to-zero mode turned on.

Note:
the Oracle code cannot be changed or compiled such that the condition that generates the message is eliminated.
This because the hardware doesn't support the floating-point operation being reque

Solution

It is possible to completely turn off the floating-point assist messages from the console (they are still written to /var/log/messages)
To do this, simply issue the command as "root":

$ dmesg -n4

To eliminate them also from /var/log/messages :

1. edit /etc/syslog.conf changing line:

*.info;mail.none;authpriv.none;cron.none /var/log/messages

*.error;mail.none;authpriv.none;cron.none /var/log/messages

This means any facility logging messages below error level will be suppressed

2. restart syslog
service syslog restart

To explain wht is being suppressed:
From the syslog man page:

Every text line in a message has its own loglevel.
This level is DEFAULT_MESSAGE_LOGLEVEL - 1 (6) unless the line starts with <d> where d is a digit in the range 1-7,
in which case the level is d. The conventional meaning of the loglevel is defined in <linux/kernel.h> as follows:

       #define KERN_EMERG    "<0>" /* system is unusable               */
       #define KERN_ALERT    "<1>" /* action must be taken immediately */
       #define KERN_CRIT     "<2>" /* critical conditions              */
       #define KERN_ERR      "<3>" /* error conditions                 */
       #define KERN_WARNING "<4>" /* warning conditions               */
       #define KERN_NOTICE   "<5>" /* normal but significant condition */
       #define KERN_INFO     "<6>" /* informational                    */
       #define KERN_DEBUG    "<7>" /* debug-level messages             */

The n4 in the dmesg command means to filter messages of "warning conditions" of which the floating point software assist messages are. The *.error in syslog.conf means all levels below e

References

BUG:3777000 - FLOATING-POINT ASSIST FAULT(FPSWA) CAUSES POOR PERFORMANCE
BUG:3796598 - KERNEL: ORACLE(570): FLOATING-POINT ASSIST FAULT AT IP CAUSES CONNECTION PROBLEM
http://i-cluster2.inrialpes.fr/doc/misc/fpswa.txt
http://bugpriority.oraclecorp.com/pls/bugpriority/DWB_RFA_HTML.RFA_ADMIN?P_RFA_REQUEST_ID=1158
http://h21007.www2.hp.com/portal/site/dspp/menuitem.863c3e4cbcdc3f3515b49c108973a801?ciid=62080055abe021100055abe02110275d6e10RCRD

Related

Products

Oracle Database Products > Oracle Database > Oracle Database > Oracle Server - Enterprise Edition

DBCC CHECKDB

2010-02-17T00:56:00.002+07:00

Checks the allocation and structural integrity of all the objects in the specified database.

Syntax

DBCC CHECKDB
    ( 'database_name'
            [ , NOINDEX
                | { REPAIR_ALLOW_DATA_LOSS
                    | REPAIR_FAST
                    | REPAIR_REBUILD
                    } ]
    )    [ WITH { [ ALL_ERRORMSGS ]
                    [ , [ NO_INFOMSGS ] ]
                    [ , [ TABLOCK ] ]
                    [ , [ ESTIMATEONLY ] ]
                    [ , [ PHYSICAL_ONLY ] ]
                    }
        ]

Arguments

'database_name'

Is the database for which to check all object allocation and structural integrity. If not specified, the default is the current database. Database names must conform to the rules for identifiers. For more information, see Using Identifiers [ http://msdn.microsoft.com/en-us/library/aa223962(SQL.80).aspx ] .

NOINDEX

Specifies that nonclustered indexes for nonsystem tables should not be checked. NOINDEX decreases the overall execution time because it does not check nonclustered indexes for user-defined tables. NOINDEX has no effect on system tables, because DBCC CHECKDB always checks all system table indexes.

REPAIR_ALLOW_DATA_LOSS | REPAIR_FAST| REPAIR_REBUILD

Specifies that DBCC CHECKDB repair the found errors. The given database_name must be in single-user mode to use a repair option and can be one of the following.

Value	Description
REPAIR_ALLOW_DATA_LOSS	Performs all repairs done by REPAIR_REBUILD and includes allocation and deallocation of rows and pages for correcting allocation errors, structural row or page errors, and deletion of corrupted text objects. These repairs can result in some data loss. The repair may be done under a user transaction to allow the user to roll back the changes made. If repairs are rolled back, the database will still contain errors and should be restored from a backup. If a repair for an error has been skipped due to the provided repair level, any repairs that depend on the repair are also skipped. After repairs are completed, back up the database.
REPAIR_FAST	Performs minor, nontime-consuming repair actions such as repairing extra keys in nonclustered indexes. These repairs can be done quickly and without risk of data loss.
REPAIR_REBUILD	Performs all repairs done by REPAIR_FAST and includes time-consuming repairs such as rebuilding indexes. These repairs can be done without risk of data loss.

WITH

Specifies options on the number of error messages returned, locks obtained, or estimating tempdb requirements.

ALL_ERRORMSGS

Displays an unlimited number of errors per object. If ALL_ERRORMSGS is not specified, displays up to 200 error messages for each object. Error messages are sorted by object ID, except for those messages generated from tempdb.

NO_INFOMSGS

Suppresses all informational messages (Severity 10) and the report of space used.

TABLOCK

Causes DBCC CHECKDB to obtain shared table locks. TABLOCK will cause DBCC CHECKDB to run faster on a database under heavy load, but decreases the concurrency available on the database while DBCC CHECKDB is running.

ESTIMATE ONLY

Displays the estimated amount of tempdb space needed to run DBCC CHECKDB with all of the other specified options. The check is not performed.

PHYSICAL_ONLY

Limits the checking to the integrity of the physical structure of the page and record headers, and to the consistency between the pages' object ID and index ID and the allocation structures. Designed to provide a low overhead check of the physical consistency of the database, this check also detects torn pages and common hardware failures that can compromise a user's data. PHYSICAL_ONLY always implies NO_INFOMSGS and is not allowed with any of the repair options.

Remarks

DBCC CHECKDB performs a physical consistency check on indexed views. The NOINDEX option, used only for backward compatibility, also applies to any secondary indexes on indexed views.

DBCC CHECKDB is the safest repair statement because it identifies and repairs the widest possible errors. If only allocation errors are reported for a database, execute DBCC CHECKALLOC with a repair option to repair these errors. However, to ensure that all errors, including allocation errors, are properly repaired, execute DBCC CHECKDB with a repair option rather than DBCC CHECKALLOC with a repair option.

DBCC CHECKDB validates the integrity of everything in a database. There is no need to run DBCC CHECKALLOC or DBCC CHECKTABLE if DBCC CHECKDB either is currently or has been recently executed.

DBCC CHECKDB performs the same checking as if both a DBCC CHECKALLOC statement and a DBCC CHECKTABLE statement were executed for each table in the database.

DBCC CHECKDB does not acquire table locks by default. Instead, it acquires schema locks that prevent meta data changes but allow changes to the data. The schema locks acquired will prevent the user from getting an exclusive table lock required to build a clustered index, drop any index, or truncate the table.

The DBCC statement collects information, and then scans the log for any additional changes made, merging the two sets of information together to produce a consistent view of the data at the end of the scan.

When the TABLOCK option is specified, DBCC CHECKDB acquires shared table locks. This allows more detailed error messages for some classes of errors and minimizes the amount of tempdb space required by avoiding the use of transaction log data. The TABLOCK option will not block the truncation of the log and will allow the command to run faster.

DBCC CHECKDB checks the linkages and sizes of text, ntext, and image pages for each table, and the allocation of all the pages in the database.

For each table in the database, DBCC CHECKDB checks that:

Index and data pages are correctly linked.
Indexes are in their proper sort order.
Pointers are consistent.
The data on each page is reasonable.
Page offsets are reasonable.

Errors indicate potential problems in the database and should be corrected immediately.

By default, DBCC CHECKDB performs parallel checking of objects. The degree of parallelism is determined automatically by the query processor. The maximum degree of parallelism is configured in the same manner as that of parallel queries. Use the sp_configure system stored procedure to restrict the maximum number of processors available for DBCC checking. For more information, see max degree of parallelism Option [ http://msdn.microsoft.com/en-us/library/aa196725(SQL.80).aspx ] .

Parallel checking can be disabled by using trace flag 2528. For more information, see Trace Flags [ http://msdn.microsoft.com/en-us/library/aa260630(SQL.80).aspx ] .

Result Sets

Whether or not any options (except for the NO_INFOMSGS or NOINDEX options) are specified, DBCC CHECKDB returns this result set for the current database, if no database is specified (values may vary):

DBCC results for 'master'.
DBCC results for 'sysobjects'.
There are 862 rows in 13 pages for object 'sysobjects'.
DBCC results for 'sysindexes'.
There are 80 rows in 3 pages for object 'sysindexes'.
'...'
DBCC results for 'spt_provider_types'.
There are 23 rows in 1 pages for object 'spt_provider_types'.
CHECKDB found 0 allocation errors and 0 consistency errors in database 'master'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

IF the NO_INFOMSGS option is specified, DBCC CHECKDB returns this result set (message):

The command(s) completed successfully.

DBCC CHECKDB returns this result set when the ESTIMATEONLY option is specified.

Estimated TEMPDB space needed for CHECKALLOC (KB) 
------------------------------------------------- 
13

(1 row(s) affected)

Estimated TEMPDB space needed for CHECKTABLES (KB) 
-------------------------------------------------- 
57

(1 row(s) affected)

DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Permissions

DBCC CHECKDB permissions default to members of the sysadmin fixed server role or the db_owner fixed database role, and are not transferable.

Examples

A. Check both the current and the pubs database

This example executes DBCC CHECKDB for the current database and for the pubs database.

-- Check the current database.
DBCC CHECKDB
GO
-- Check the pubs database without nonclustered indexes.
DBCC CHECKDB ('pubs', NOINDEX)
GO

B. Check the current database, suppressing informational messages

This example checks the current database and suppresses all informational messages.

DBCC CHECKDB WITH NO_INFOMSGS
GO

DBCC CHECKALLOC (Transact-SQL)

2010-02-17T00:54:00.002+07:00

Updated: 17 November 2008

Checks the consistency of disk space allocation structures for a specified database.

Transact-SQL Syntax Conventions [ http://technet.microsoft.com/en-us/library/ms177563(SQL.90).aspx ]

Syntax

DBCC CHECKALLOC

[         ( database_name | database_id | 0       [ , NOINDEX       | , { REPAIR_ALLOW_DATA_LOSS | REPAIR_FAST | REPAIR_REBUILD } ]         )     [ WITH         {           [ ALL_ERRORMSGS ]           [ , NO_INFOMSGS ]           [ , TABLOCK ]           [ , ESTIMATEONLY ]         }     ]

]

Arguments

database_name | database_id | 0

Is the name or the ID of the database for which to check allocation and page usage. If not specified, or if 0 is specified, the current database is used.

Database names must follow the rules for identifiers [ http://technet.microsoft.com/en-us/library/ms175874(SQL.90).aspx ] .

NOINDEX

Specifies that nonclustered indexes for user tables should not be checked.

Note:
NOINDEX is maintained for backward compatibility only and does not affect DBCC CHECKALLOC.

REPAIR_ALLOW_DATA_LOSS | REPAIR_FAST | REPAIR_REBUILD

Specifies that DBCC CHECKALLOC repair the found errors. database_name must be in single-user mode.

REPAIR_ALLOW_DATA_LOSS: Tries to repair any errors that are found. These repairs can cause some data loss. REPAIR_ALLOW_DATA_LOSS is the only option that allows for allocation errors to be repaired.

REPAIR_FAST: Syntax is maintained for backward compatibility only. No repair actions are performed.

REPAIR_REBUILD: Not applicable.

Important:
Use the REPAIR options only as a last resort. To repair errors, we recommend restoring from a backup. Repair operations do not consider any of the constraints that may exist on or between tables. If the specified table is involved in one or more constraints, we recommend running DBCC CHECKCONSTRAINTS after a repair operation. If you must use REPAIR, run DBCC CHECKDB without a repair option to find the repair level to use. If you use the REPAIR_ALLOW_DATA_LOSS level, we recommend that you back up the database before you run DBCC CHECKDB with this option.

Important:

Use the REPAIR options only as a last resort. To repair errors, we recommend restoring from a backup. Repair operations do not consider any of the constraints that may exist on or between tables. If the specified table is involved in one or more constraints, we recommend running DBCC CHECKCONSTRAINTS after a repair operation. If you must use REPAIR, run DBCC CHECKDB without a repair option to find the repair level to use. If you use the REPAIR_ALLOW_DATA_LOSS level, we recommend that you back up the database before you run DBCC CHECKDB with this option.

WITH: Enables options to be specified.

ALL_ERRORMSGS: Displays all error messages. In SQL Server 2005 Service Pack 3 (SP3), all error messages are displayed by default. Specifying or omitting this option has no effect. In earlier versions of SQL Server, only the first 200 error messages for each object are displayed if ALL_ERRORMSGS is not specified.

NO_INFOMSGS: Suppresses all informational messages and the report of space used.

TABLOCK: Causes the DBCC command to obtain an exclusive database lock.

ESTIMATE ONLY: Displays the estimated amount of tempdb space that is required to run DBCC CHECKALLOC when all the other options are specified.

Result Sets

The following tables describe the information that DBCC CHECKALLOC returns.

Item	Description
FirstIAM	Internal use only.
Root	Internal use only.
Dpages	Data page count.
Pages used	Allocated pages.
Dedicated extents	Extents allocated to the object. If mixed allocation pages are used, there might be pages allocated without extents.

DBCC CHECKALLOC also reports an allocation summary for each index and partition in each file. This summary describes the distribution of the data.

Item	Description
Reserved pages	Pages allocated to the index and the unused pages in allocated extents.
Used pages	Pages allocated and being used by the index.
Partition ID	Internal use only.
Alloc unit ID	Internal use only.
In-row data	Pages contain index or heap data.
LOB data	Pages contain varchar(max), nvarchar(max), varbinary(max), text, ntext, xml, and image data.
Row-overflow data	Pages contain variable-length column data that has been pushed off-row.

DBCC CHECKALLOC returns the following result set (values may vary), except when ESTIMATEONLY or NO_INFOMSGS is specified.

DBCC results for 'master'.

***************************************************************

Table sysobjects                Object ID 1.

Index ID 1         FirstIAM (1:11)   Root (1:12)    Dpages 22.

   Index ID 1. 24 pages used in 5 dedicated extents.

Index ID 2         FirstIAM (1:1368)   Root (1:1362)    Dpages 10.

   Index ID 2. 12 pages used in 2 dedicated extents.

Index ID 3         FirstIAM (1:1392)   Root (1:1408)    Dpages 4.

   Index ID 3. 6 pages used in 0 dedicated extents.

Total number of extents is 7.

***************************************************************

'...'

***************************************************************

Table spt_server_info                Object ID 1938105945.

Index ID 1         FirstIAM (1:520)   Root (1:508)    Dpages 1.

   Index ID 1. 3 pages used in 0 dedicated extents.

Total number of extents is 0.

***************************************************************

Processed 52 entries in sysindexes for database ID 1.

File 1. Number of extents = 210, used pages = 1126, reserved pages = 1280.

          File 1 (number of mixed extents = 73, mixed pages = 184).

   Object ID 1, Index ID 0, data extents 5, pages 24, mixed extent pages 9.

'...'

   Object ID 1938105945, Index ID 0, data extents 0, pages 3, mixed extent pages 3.

Total number of extents = 210, used pages = 1126, reserved pages = 1280 in this database.

      (number of mixed extents = 73, mixed pages = 184) in this database.

CHECKALLOC found 0 allocation errors and 0 consistency errors in database 'master'.

DBCC results for 'master'.

***************************************************************

Table sys.sysrowsetcolumns                Object ID 4.

Index ID 1, partition ID 262144, alloc unit ID 262144 (type In-row data). FirstIAM (1:98). Root (1:94). Dpages 7.

Index ID 1, partition ID 262144, alloc unit ID 262144 (type In-row data). 9 pages used in 1 dedicated extents.

Index ID 1, partition ID 262144, alloc unit ID 262398 (type Row-overflow data). FirstIAM (0:0). Root (0:0). Dpages 0.

Index ID 1, partition ID 262144, alloc unit ID 262398 (type Row-overflow data). 0 pages used in 0 dedicated extents.

Total number of extents is 1.

...

***************************************************************

Processed 201 entries in system catalog for database ID 1.

File 1. Number of extents = 44, used pages = 300, reserved pages = 345.

          File 1 (number of mixed extents = 29, mixed pages = 225).

   Object ID 4, index ID 1, partition ID 262144, alloc unit ID 262144 (type In-row data), data extents 1, pages 9, mixed extent pages 8.

   Object ID 5, index ID 1, partition ID 327680, alloc unit ID 327680 (type In-row data), data extents 0, pages 2, mixed extent pages 2.

   Object ID 7, index ID 1, partition ID 458752, alloc unit ID 458752 (type In-row data), data extents 0, pages 5, mixed extent pages 5.

   Object ID 8, index ID 0, partition ID 524288, alloc unit ID 524288 (type In-row data), data extents 0, pages 2, mixed extent pages 2.

   Object ID 13, index ID 1, partition ID 851968, alloc unit ID 851968 (type In-row data), data extents 1, pages 9, mixed extent pages 8.

   Object ID 15, index ID 1, partition ID 983040, alloc unit ID 983040 (type In-row data), data extents 0, pages 2, mixed extent pages 2.

   Object ID 26, index ID 1, partition ID 281474978414592, alloc unit ID 1703937 (type In-row data), data extents 0, pages 3, mixed extent pages 3.

   Object ID 27, index ID 1, partition ID 281474978480128, alloc unit ID 1769473 (type In-row data), data extents 0, pages 3, mixed extent pages 3.

   Object ID 27, index ID 2, partition ID 562949955190784, alloc unit ID 1769474 (type In-row data), index extents 0, pages 3, mixed extent pages 3.

...

   Object ID 1179151246, index ID 1, partition ID 72057594038845440, alloc unit ID 13435136 (type In-row data), data extents 2, pages 18, mixed extent pages 8.

   Object ID 1179151246, index ID 2, partition ID 72057594038910976, alloc unit ID 13566208 (type In-row data), index extents 1, pages 16, mixed extent pages 8.

   Object ID 1911677858, index ID 0, partition ID 72057594039631872, alloc unit ID 15073536 (type In-row data), data extents 0, pages 2, mixed extent pages 2.

Total number of extents = 41, used pages = 289, reserved pages = 323 in this database.

      (number of mixed extents = 27, mixed pages = 211) in this database.

CHECKALLOC found 0 allocation errors and 0 consistency errors in database 'master'.

DBCC execution completed. If DBCC printed error messages, contact your system administrator.

When ESTIMATEONLY is specified, DBCC CHECKALLOC returns the following result set.

Estimated TEMPDB space needed for CHECKALLOC (KB)

-------------------------------------------------

34



(1 row(s) affected)



DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Remarks

DBCC CHECKALLOC checks the allocation of all pages in the database, regardless of the type of page or type of object to which they belong. It also validates the various internal structures that are used to keep track of these pages and the relationships between them.

If NO_INFOMSGS is not specified, DBCC CHECKALLOC collects space usage information for all objects in the database. This information will then be printed together with any errors that are found.

Note:
The DBCC CHECKALLOC functionality is included in DBCC CHECKDB [ http://technet.microsoft.com/en-us/library/ms176064(SQL.90).aspx ] and DBCC CHECKFILEGROUP [ http://technet.microsoft.com/en-us/library/ms187332(SQL.90).aspx ] . This means that you do not have to run DBCC CHECKALLOC separately from these statements.

Internal Database Snapshot

DBCC CHECKALLOC uses an internal database snapshot to provide the transactional consistency that it needs to perform these checks. If a snapshot cannot be created, or TABLOCK is specified, DBCC CHECKALLOC tries to acquire an exclusive (X) lock on the database to obtain the required consistency. For more information about locks, see Lock Modes [ http://technet.microsoft.com/en-us/library/ms175519(SQL.90).aspx ] .

Note:
In SQL Server 2005, running DBCC CHECKALLOC against tempdb does not perform any checks. This is because, for performance reasons, database snapshots are not available on tempdb. This means that the required transactional consistency cannot be obtained. Stop and start the MSSQLSERVER service to resolve any tempdb allocation issues. This action drops and re-creates the tempdb database.

Note:

In SQL Server 2005, running DBCC CHECKALLOC against tempdb does not perform any checks. This is because, for performance reasons, database snapshots are not available on tempdb. This means that the required transactional consistency cannot be obtained. Stop and start the MSSQLSERVER service to resolve any tempdb allocation issues. This action drops and re-creates the tempdb database.

Understanding DBCC Error Messages

After the DBCC CHECKALLOC command finishes, a message is written to the SQL Server error log. If the DBCC command successfully executes, the message indicates a successful completion and the amount of time that the command ran. If the DBCC command stops before completing the check because of an error, the message indicates the command was terminated, a state value, and the amount of time the command ran. The following table lists and describes the state values that can be included in the message.

State	Description
0	Error number 8930 was raised. This indicates a metadata corruption that caused the DBCC command to terminate.
1	Error number 8967 was raised. There was an internal DBCC error.
2	A failure occurred during emergency mode database repair.
3	This indicates a metadata corruption that caused the DBCC command to terminate.
4	An assert or access violation was detected.
5	An unknown error occurred that terminated the DBCC command.

Error Reporting

In SQL Server 2005 Service Pack 1 (SP1), a mini-dump file (SQLDUMPnnnn.txt) is created in the SQL Server LOG directory whenever DBCC CHECKALLOC detects a corruption error. When the Feature Usage data collection and Error Reporting features are enabled for the instance of SQL Server, the file is automatically forwarded to Microsoft. The collected data is used to improve SQL Server functionality. For more information, see Error and Usage Report Settings [ http://technet.microsoft.com/en-us/library/ms144283(SQL.90).aspx ] .

The dump file contains the results of the DBCC CHECKALLOC command and additional diagnostic output. The file has restricted discretionary access-control lists (DACLs). Access is limited to the SQL Server service account and members of the sysadmin role. By default, the sysadmin role contains all members of the Windows BUILTIN\Administrators group and the local administrator's group. The DBCC command does not fail if the data collection process fails.

Resolving Errors

If DBCC CHECKALLOC reports any errors, we recommend that you restore the database from the database backup instead of running a repair. If a backup does not exist, running a repair can correct the reported errors; however, correcting the errors might require some pages, and therefore data, to be deleted.

A repair can be performed in a user transaction. This allows for changes to be rolled back. If changes are rolled back, the database will still contain errors and must be restored from a backup. After the repairs finish, back up the database.

Permissions

Requires membership in the sysadmin fixed server role or the db_owner fixed database role.

Examples

The following example executes DBCC CHECKALLOC for the current database and for the AdventureWorks database.

-- Check the current database.

DBCC CHECKALLOC;

GO

-- Check the AdventureWorks database.

DBCC CHECKALLOC (AdventureWorks);

GO

Reference

DBCC (Transact-SQL) [ http://technet.microsoft.com/en-us/library/ms188796(SQL.90).aspx ]

Other Resources

Table and Index Organization [ http://technet.microsoft.com/en-us/library/ms189051(SQL.90).aspx ]

Managing Space Used by Objects [ http://technet.microsoft.com/en-us/library/ms187501(SQL.90).aspx ]

Help and Information

Getting SQL Server 2005 Assistance [ http://technet.microsoft.com/en-us/library/ms166016(SQL.90).aspx ]

Change History

Release	History
17 November 2008	New content: In the definition of ALL_ERRORMSGS, described new functionality in SP3.
14 April 2006	New content: In the Remarks section, added the subsection "Error Reporting." This section describes new functionality in SP1.
5 December 2005	Changed content: Corrected the syntax. Added xml to the list of data types in "LOB Data." Corrected the REPAIR_FAST definition. The option does not perform repair actions. Added to the Remarks section information about messages that are written to the SQL Server error log and descriptions of state values.

Tags:

Community Content

NLS Guide for Generic Connectivity and Gateways [ID 230239.1]

2010-02-16T11:57:00.001+07:00



		Modified 18-DEC-2008 Type BULLETIN Status PUBLISHED

PURPOSE
-------

Analysing NLS issues when HS or Gateways are involved.

SCOPE & APPLICATION
-------------------

How to analyse NLS related issues with Gateways or Generic Connectivity.

How to solve NLS related problems with generic connectivity or gateways
-----------------------------------------------------------------------

This note is devided into four parts. The first part is some useful
background information. Part two is a description of the language
parameter, part three is a guide how to solve decimal
separator or lost decimal issues and part four describes steps
to check why characters are not displayed correctly.

I. Background Information
#########################
Each database works with a special character set, language and
territory setting.

Ideally, the character set of the Oracle database server
and the non-Oracle data source are the same. Then no
conversion during data transfer is needed.
If the character set differs, Heterogeneous Services / Gateways must
translate the character set of the non-Oracle data source
to the Oracle database character set and back again.

To be able to translate between two kind of things, you must know
the source and the target data. This is true for translating from
one language to another as well as for currency ...

The Generic connectivity/Gateway translates characters and
territory settings from the remote database to the Oracle database
and vice versa.
To do so, it must be aware of the remote character and territory settings.
A parameter called HS_LANGUAGE could be set in the initialisation file
of the gateway to reflect the REMOTE settings.
HS_LANGUAGE provides Heterogeneous Services / Gateways with character set,
language and territory information of the non-Oracle data source.
While the remote settings are specified in the initialisation file
and the local settings are defined with the NLS_LANG parameter a
conversation should be successful.

HS_LANGUAGE=language[_territory.character_set]

II. Language settings and their relationship
############################################

The language part of the HS_LANGUAGE initialization parameter
determines:

Day and month names of dates
AD, BC, PM, and AM symbols for date and time
Default sorting mechanism

Note: HS_LANGUAGE does not determine the language for error messages.

III. Truncated number columns or lost decimals
##############################################

Decimal delimiters depend on country specific settings. Some
countries use a comma, others a point to separate decimals.
Local and remote databases can work with different decimal delimiters.
Without being aware of the remote decimal delimiter it might happen
that decimals are lost during conversation.

Example:

- Microsoft SQL Server with American NLS settings
- Oracle Server 9.2.0.1 with German NLS settings
- GERMAN_GERMANY.WE8MSWIN1252.
- Derived NLS_NUMERIC_CHARACTERS from German NLS_TERRITORY is ", ."

MS SQL Server with a table num and 2 columns:
a as numeric with a value 8.5
and b as char with a string 'd'
in it

SQL>select * from num@sql7;

a b
---------- ---------------
8 d

In this sample everything beyond the decimal point is truncated.
The reason is that the SQL*Plus session works with GERMAN
territory settings, and inherits this setting to the HS.
MS SQL Server works with American settings. Both regions use
different decimal delimiters - German schema uses ','; American
schema '.'-.

To resolve this issue the parameter HS_LANGUAGE in
the initialisation file of the gateway must be
changed to AMERICAN_AMERICA.WE8MSWIN1252 to reflect the
REMOTE NLS settings.

SQL>select * from num@sql7;

a b
---------- ---------------
8,5 d

Sometimes it's hard to figure out the real Territory of the
foreign database; in might be changed due to the client settings.
So another simple approach is to vary the HS_LANGUAGE parameter
and try it:
For the first try set the HS_Language to
AMERICAN_AMERICA.we8iso8859p1. This will work for foreign data stores
having a POINT as decimal delimeter. After changing any HS parameter,
please make sure you close the old connection. The new parameter will be
picked up only during the connection phase.
If you still won't be able to get the correct values back, now try
HS_LANGUAGE with an European character set like GERMAN_GERMANY.we8iso8859p1

In almost all cases the decimals could be displayed with method 1 or 2
correctly.

IV. Characters are not displayed correctly
##########################################

A) Single Byte characters
A code page, also known as character set, is a set of 256
uppercase and lowercase letters, numbers and symbols. The printable
characters of the first 128 values are the same for all character set
choices. The last 128 characters differ from set to set. All those 256
characters are single byte characters (0x00-0xFF).

Before analysing any details, make sure that the operating system
is capable of dealing with these characters:
Microsoft Windows NT contains an ASCII table.
The program is called charmap.exe and located %SystemRoot%\System32\.

Let's check out the ARIAL font; it contains for example a character
'Ã„' (0xC4).
To make sure, that the operating system can display this character 'Ã„',
copy it from this Ascii table and paste it into NOTEPAD.

Is it displayed correctly?
If not, you have to figure out the language YOUR character belongs to
and the Windows environment must be changed accordingly.
This is done by changing the regional settings of the operating system
( -> Start -> Settings -> Control Panel -> Regional Settings).

Once you figured out the correct settings, proceed to the next step. Do NOT
proceed until you're able to display YOUR character correctly in NOTEPAD!
If you have problems to figure out the correct settings, please contact
Microsoft Support.

The next step is to verify that SQL*Plus is capable displaying the
characters.

SQL*PLUS GUI (sqlplusw):
------------------------
At this stage we don't need a connection to the database. Start a GUI
SQL*Plus session with the nolog option.
sqlplusw /nolog

Side Note: Everything is done with the GUI based SQL*Plus.
Inserting from the Windows command line tool and selecting the
inserted characters from the GUI based SQL*Plus could cause problems,
because the DOS Box character set ( 437 or 850...) differs from the GUI
character set and characters might be interpreted wrongly.

From the ASCII table choose again YOUR character and copy paste it now into
the SQL*Plus session. Does it display correctly?

If not, there are a few parameters to change the FONT and CHARACTER set
of SQL*Plus:
The most important registry values are
SQLPLUS_FONT and SQLPLUS_FONT_CHARSET.
Both are string values (REG_SZ) and
must be added manually to the HKEY_LOCAL_MACHINE/SOFTWARE/ORACLE/HOMEx key
where x is the number of your Oracle_Home installation associated with
SQL*Plus was called from. These values are read each time SQL*Plus starts.

SQLPLUS_FONT:
Any fixed-pitch TrueType font available in Windows system such as
Courier New or Lucida Console can be used. Choosing a proportional
pitch font like Arial or Times New Roman, or entering an unavailable
font, the registry entry is ignored and the default font
Fixedsys 16, is used.

Prior Oracle 9iR2 this was hard work: A capable font that is used
by SQL*Plus and that also contains the character that should be
displayed must be figured out.

Oracle 9iR2 introduced the SQLPLUS_FONT_CHARSET and with this value it
becomes easier. The SQLPLUS_FONT can now be the default value and the
character set is managed by the
SQLPLUS_FONT_CHARSET
registry entry.
It defines the font subset used in the SQL*Plus Windows GUI.
The following values are valid:
DEFAULT
SYMBOL
SHIFTJIS
HANGEUL
GB2312
CHINESEBIG5
OEM
JOHAB
HEBREW
ARABIC
GREEK
TURKISH
VIETNAMESE
THAI
EASTEUROPE
RUSSIAN
MAC
BALTIC

While choosing one of those from the list, it should be possible to
display the character in SQL*Plus.
This step is also a preleminary step and you must not proceed before this
works.

SQL*Plus COMMAND LINE (sqlplus):
To configure the command line version of SQL*Plus please open a DOS command line
window and check out the PROPERTIES of this window. The font needs to be a TRUE
TYPE font like "Lucida Console", the default RASTER FONT is not able to display
any foreign character.

Next step is to check out the CODE PAGE of the shell by typing
"chcp" (chcp =>change codepage and it will display the active code page).
Please make sure that this code page is able to display your characters; else
change the code page to a code page that covers your characters. More
details can be found at Microsoft's Web site.
Last step is now to set NLS_LANG correctly in this COMMAND Line window.
There is NO need to change the registry key at all.

By now, the OS and SQL*Plus can display the character.
The next questions are:
Is the character inserted correctly in the remote database?
Is the Oracle database capable to deal with this character?
Is HS/Gateway capable to deal with this character?

Figuring out the correct ASCII value for a character in the remote table
depends on the database vendor:

AS/400 servers for example:
'DSPFFD' to get the codepage of the column/table and use
select hex(<col>) from table to get the dump values.
To insert dediacted hex values into an AS400 table the function char(X'<hex value>')
like char(X'b8') can be used.

MS SQL Server for example provides the ASCII function:
select ascii(<column name>) from <table>

Demonstration of the ASCII function of MS SQL Server:
select ascii('T')
reports
84
This output is the decimal value for this character 'T'; 84 decimal is the
same as 54 hexadecimal and this is accoring to the charmap.exe the capital
T.

While the inserted characters correspond to the correct values in the
charmap, the next step is to verify the capability of the Oracle database:

select dump('T') from dual;
results in:
DUMP('T')
----------------
Typ=96 Len=1: 84

The important infomation is '84'. This is decimal and represents the
'T' letter.

Both methods (dumping the remote and the Oracle side) to verify the
decimal value for a character resolve the same result.

Side Note:
Sometimes also the application might insert characters incorrectly. Thus insert the
character with SQL*Plus into a test table and afterwards again with the
application. Then compare both dumps.

The last step now is to verify the HS sub system:
Only after all the steps above are successful, the HS system should be
checked. HS depends on a well configured remote and local database.

The simplest way to verify the HS is again a test table at the remote
database with test characters inserted. Especially use one character
out of the first 128 characters from the character map. They are the
same for all character maps and this can be very helpful.
Then also insert the characters that are not being displayed correctly.

Then create the same table in the Oracle database and fill it with the
same records.

Now dump the characters of the Oracle db and also from the remote db.
select dump(<column>) from <table>@<db_link>;
select dump(<column>) from <table>;

Are the decimal values the same?
If the dumped value is the same, but the SQL*Plus output differs,
then it looks like HS is wrongly initialized.
Check out the initialisation file of the HS/Gateway and verify
the settings of HS_LANGUAGE:
The HS_LANGUAGE initialization parameter provides Heterogeneous
Services with character set, language and territory information
of the non-Oracle data source. The value of the HS_LANGUAGE
initialization parameter has to be of the following format:

<language>[_<territory>.<character_set>]

Character_set:
This should be set to the REMOTE DATABASE CHARACTER SET or
to a super set of the remote database character set.

While the character set specifies the remote database character set,
all parameters (language and territory) must be initialized!

Language:
Day and month names of dates
AD, BC, PM, and AM symbols for date and time
Default sorting mechanism

Territory:
The territory clause of the HS_LANGUAGE initialization
parameter specifies the conventions for day and week numbering,
default date format, decimal character and group separator,
and ISO and local currency symbols.

Important:
The level of National Language Support between the Oracle server
and the non-Oracle data source depends on how the driver is implemented.
See the installation and users' guide for your platform for more
information about the level of National Language Support.

B)MULTI BYTE Characters:
The same from above is also valid for MULTI BYTE as well as UNICODE
characters.
The only difference is that the dump will differ:

Example for dumping unicode characters of a MS SQL Server:
insert into <table> values (nchar(30333)...)
select unicode(<column>) from table

This value belongs to a chinese character set; thus setting the
NLS_LANG of the client to ZHT16BIG5 and dumping now this remote table
with SQL*Plus and Oracle 9i onwards:
select asciistr(<column>) from <table>@<db_link>;
results in:
ASCIISTR("B")
----------------------
\767D

The Hex value 767D is equivalent to decimal 30333 we have inserted.

To verify the retrieved value matches the same character in an Oracle db a
select unistr('\767D') from dual; should show up the same character as
it is stored in the foreign database.

If the dump values differ between the source and the target database,
make sure the NLS_LANG is set to at least a character set that contains
the remote character. If the character set match, the next reason could be
the HS subsytem not being aware of the UNICODE characters.
The initialisation file of the gateway/hs knows a parameter called:

HS_NLS_NCHAR
and this parameter must refelct the REMOTE NATIONAL CHARACTER SET;
for MS SQL Server for example: UCS2
=> HS_NLS_NCHAR=UCS2

Additional comment:
Instead of configuring SQL*Plus another approach would be to use Microsoft
Word (if installed on the machine) and use the MS Word unicode support.

There is NO need to configure SQL*Plus at all. The following procedure will
perform a select using the gateway or HSODBC and write the output directly into
a file using the unicode compliant utl_file function.
Then just open the DOC file output using MS Word and the preview will display
the characters within the DOC:

Make sure the following directory exists or adapt it according to your
preferences:
d:\test_utf8

Now log into the database and prepare UTL_FILE function:

create or replace directory TEST_UTF8 as 'd:\test_utf8';
grant read, write on directory TEST_UTF8 to public;

Assuming now we have a table at the SQL Server database called
"russian" and "col3" conatins the russian characters:

CREATE OR REPLACE procedure testUTF8_4 as
v_file UTL_FILE.FILE_TYPE;
line_org varchar2(2000);
line_conv varchar2(2000);
v_datensatz varchar2(20000);
Cursor c1 is
Select "col3" from "russia"@sqlserver;

begin
v_file := UTL_FILE.FOPEN_nchar('TEST_UTF8','RUSSIAN.doc', 'w');
open c1;
LOOP
fetch c1 into v_datensatz;
Exit when c1%notFOUND;

UTL_FILE.PUT_LINE_nchar(v_file, v_datensatz);
UTL_FILE.NEW_LINE (v_file);
end loop;
close c1;
UTL_FILE.FClose(v_file);
DBMS_OUTPUT.PUT_LINE('DONE!');
End;

This procedure does not depend on the SQL*Plus settings. So there is no
need to change the registry keys. It is using the database settings and writes a
file to d:\test_utf8 called RUSSIAN.doc. Now open this doc file using MS Word
and MS Word is displaying a preview Window where you can choose the correct
character set by simply selecting it from a list and depending on the selection
the preview window will change.

To get rid of the UTL_FILE_DIR configuration please execute:
drop directory TEST_UTF8;

How to setup HSODBC on HP-UX [ID 270395.1]

2010-02-16T11:47:00.001+07:00



		Modified 28-JUL-2008 Type BULLETIN Status ARCHIVED


PURPOSE
-----------------------------------------------------------------
PLEASE BE AWARE THAT THERE IS NO PORTING OF GENERIC CONNECTIVITY
FOR HP-UX ITANIUM.

This note describes only setting up HSODBC on HP-UX RISC platforms.

 
SCOPE & APPLICATION
-------------------
This note describes how to set up HSODBC (heterogeneous services) for
HP-UX 11.0 using Oracle 9.2 release. It is similar for 9.1 release, but
the directory structure changed between 8i and 9i, so while using this note
for 8i, please keep in mind to change the directories of the libraries.



How to Setup HSODBC (heterogeneous services) on HP-UX
-----------------------------------------------------
 
This note is divided into five different parts:
Part I describes how to set up the ODBC driver
Part II describes how to relink the HSODBC executable
Part III describes the configuration process of HSODBC
Part IV is a summary of how to set it up for Oracle 8i on HP-UX
Part V Comments


Part I: Setting up the ODBC driver
----------------------------------
There are several ODBC vendors for UNIX platforms.
This note describes the settings of the ODBC driver from
Data Direct Technologies. They also provide a 15 day trial 
license for HP-UX platforms. More information is available at 
the following URL:
http://www.datadirect-technologies.com

HSODBC is a 32 bit libray and thus it needs a 32 bit ODBC driver.
The libraries of other ODBC vendors will vary; so please make sure 
to change the libraries to the libraries of your odbc vendor.

As mentioned, Data Direct Technologies provide a trial version. 
If you download this trial version, please make sure that you download
also the service pack for the odbc driver -if it exists-.
Another feature of this driver is, that it also provides some mechanism
to check the ODBC connectivity. 

Install the ODBC driver into a separate home directory. Let's call this 
home directory ODBC_HOME (i.e. /u05/odbc) directory.
To install the driver, you can create a new user called odbc who owns 
the software.
Install the ODBC driver as mentioned in the documentation.

- Configuring the odbc.ini file:
  The odbc.ini file is similar to an address book for the odbc driver. 
  It is located by default in the ODBC_HOME directory, but can be 
  placed anywhere you like it.
  
  A side note how odbc works: The odbc driver (nothing else than a library)
  gets a request to connect to a server described in the odbc.ini file. 
  The alias for the description of the server is called:
   Data Source Name (=DSN).
  Then the driver reads the information from the odbc.ini file according to the specified
  DSN and connects to the server.
   
  The prerequisite for the odbc driver to connect to the the server is the 
  configured odbc.ini.
  It is divided into 3 different sections:
  [ODBC Data Sources]
  [<DSN>]
  [ODBC]
  
  [ODBC Data Source] is the section that contains all the available DSNs.
  [<DSN>] contains the different names of the DSNs.
  [ODBC] is the general section for the odbc driver 
  
  A very simple file of the odbc.ini file looks like:
   [ODBC Data Sources]
   mssql=DataDirect 4.10 SQL Server Wire Protocol

   [mssql]
   Driver=/u05/odbc/lib/ivmsss18.sl
   Database=PCS
   LogonID=<UID like:sa>
   Password=<password for the user>
   Address=<hostname of the SQL Server>,1433
   QuotedId=No
   AnsiNPW=No

   [ODBC]
   Trace=0
   TraceFile=/tmp/odbc.trc
   TraceDll=/u05/odbc/lib/odbctrac.sl
   InstallDir=/u05/odbc
   ConversionTableLocation=/u05/odbc/tables
   UseCursorLib=0

   The section [ODBC Data Sources] contains one datasource called mssql.
   The configuration behind the data source mssql is found in the section
   [mssql].
   It contains the address (and port) of the server to contact, the driver 
   that should be used while connecting to the remote server and the 
   user id and password of the remote server.
   The [ODBC] section contains general paremeters like tracing 
   (Trace=1 enables tracing, Trace=0 disables it).
   
After configuring the odbc.ini file, the first step is to check out if the
ODBC configuration works. Data Direct Technologies provides a demo program to 
test the connectivity and fetches some data from the remote server.
The directory $ODBC_HOME/demo contains some sql scripts to create on the foreign
database a demo table called EMP. 
For the Microsoft SQL Server for example the script is called empsqlsrv.sql.
Run this script on the SQL Server to create the EMP table. (Please make sure,
that you don't overwrite/delete any tables you need).

Before calling the program demoodbc, you need to set two environment variables:
a) SHLIB_PATH must contain the odbc library path
          export SHLIB_PATH=$ODBC_HOME/lib:$SHLIB_PATH
b) ODBCINI 
          export ODBCINI=$ODBC_HOME/odbc.ini
The odbc driver is a 32 bit program and thus the library must be added to the 
SHLIB_PATH. The ODBCINI parameter will guarantee, that the newly configured 
odbc.ini file from above is used.

Now calling the odbc demo program to query the remote MS SQL Server:
demoodbc -uid <user of the MS SQL Server> -pwd <appropriated password> <DSN>
like
demoodbc -uid sa -pwd sa mssql
should connect to the server and query the EMP table.

Please make sure, that you can successfully query the table AND don't proceed 
if this configuration fails!
If you have problems configuring the odbc driver, please contact the vendor 
of the driver.


Part II: How to relink the HSODBC executable
-------------------------------------------
(Everything done in this section is done as ORACLE user)

Most ODBC drivers are linked with a C++ compiler, but Oracle is still
compiled with a C compiler.
the HP homepage conatisn an article  
Mixing C++ with other Languages for information on linking HP aC++ modules 
with HP C
that describes in detail why the following steps are required. Please refer to
it if you need more background information.

The procedure for Oracle 9i with aC++ runtime version below A.03.30 is

Check out, if the following file exists:
/opt/aCC/lib/cpprt0_stub.o

If not, please follow this instruction to create it:
Create a file called cpprt0_stub.s with the following contents:

--- cut here ---
        .code
; stubs for static constructors in a.out
        .export __StaticCtorTable_Start,data
        .export __StaticCtorTable_End,data
__StaticCtorTable_Start
__StaticCtorTable_End

        .data
; stubs for static constructors in a.out, compiled with +z/+Z
        .export __ZStaticCtorTable_Start,data
        .export __ZStaticCtorTable_End,data
__ZStaticCtorTable_Start
__ZStaticCtorTable_End
--- cut here ---

(If you created the file on a PC, don't forget to transfer
the file in ASCII mode!)
Then compile it with the following command:
as cpprt0_stub.s -o ./cpprt0_stub.o


Now back to the relinking issue:
First rename the files hsodbc and hsodbc0 in the directory
$ORACLE_HOME/bin. Then change to the directory 
$ORACLE_HOME/rdbms/lib and copy the file cpprt0_stub.o to 
this directory.

According to HP this procedure is not longer necessary for newer aCC
versions.
 A description for Oracle 8i can be found in Note:150458.1 
Relinking Heterogeneous Service on HP-UX.

Then the environment variable HSA_ODBC_SYSLIBS must be set:
export HSA_ODBC_SYSLIBS="cpprt0_stub.o -lstd -lstream -lCsup -lcl -ldld" 
and then the relink of the hsodbc library is done as follows:
/usr/ccs/bin/make -f ins_rdbms.mk ihsodbc

The linking should proceed without any error and create the hsodbc file in
the $ORACLE_HOME/bin directory.

To make sure, that the linking worked, simply type hsodbc and press enter.
For Oracle 9.2 hsodbc executable a similar screen to the following will
appear:
ora920 @ hsodbc
 
 
Oracle Corporation --- TUESDAY   OCT 15 2002 09:31:32.746
 
Heterogeneous Agent Release 9.2.0.1.0 - Production  Built with
   Driver for ODBC


This will indicate, that the relinking was successfully. 
Please don't proceed with the next part, if this part was unsuccessful !


 
Part III: How to configure HSODBC
---------------------------------
(This section assumes that everything is done in the Oracle user account
that starts the listener!)

In general the following things must be configured:
1) listener
2) tnsnames
3) init<SID>.ora of the hs subsystem
4) environment
5) Oracle database


1) The listener needs a new SID entry like the following:
    (SID_DESC =
      (SID_NAME = hsodbc)
      (ORACLE_HOME = /u01/app/oracle/product/9.2.0)
      (PROGRAM = hsodbc)
      (ENVS=SHLIB_PATH=/u05/odbc/lib:/u01/app/oracle/product/9.2.0/lib32)
    )

Please correct the ORACLE_HOME entry and the ENVS entry.
ORACLE_HOME must point to your ORACLE_HOME directory and the ENVS
string contains entries for the SHLIB_PATH.
The minimum of the SHLIB_PATH setting must contain the Oracle library and 
the odbc library path. If the odbc driver requires the foreign data store 
client libraries (like the Progress ODBC driver), the SHLIP_PATH must contain 
this library path as well:
...
  (ENVS=SHLIB_PATH=/u05/odbc/lib:
                  /u01/app/oracle/product/9.2.0/lib32:/progress/dcl/lib)
...
(Differing from 8i releases, the 32 bit Oracle libraries are now located in the 
lib32 directory while for 8i the 32 bit library was located in the lib 
directory. So please pay attention to this while using older configuration 
files. More details can be found in Note: <109621.1>).

So a listener.ora file can look like:
SID_LIST_LISTENER920 =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = hsodbc)
      (ORACLE_HOME = /u01/app/oracle/product/9.2.0)
      (PROGRAM = hsodbc)
      (ENVS=SHLIB_PATH=/u05/odbc/lib:/u01/app/oracle/product/9.2.0/lib32)
    )
  )

LISTENER920 =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname of the Oracle Server>)
         (PORT = 1921))
      )
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC))
      )
    )
  )  

The listener.ora file contains a listener called LISTENER920 instead of the 
default name LISTENER.
To stop/start the listener from above, don't forget to set the current_listener to
listener920.

The listener must be restarted after changing the listener.ora!
  

2) The tnsnames.ora needs an entry for the HSODBC alias:
hsodbc.DE.ORACLE.COM =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname of the Oracle Server>)
       (PORT = 1921))
    )
    (CONNECT_DATA =
      (SID= hsodbc))
    (HS=OK)  
  )   

The domain of the tns alias can differ from the one used above (de.oracle.com),
depending on the parameter in the sqlnet.ora:
NAMES.DEFAULT_DOMAIN = de.oracle.com

But the important entry is the (HS=OK) key word. This key word must be added 
manually and opening the Net Configuration Assistants will remove this 
entries from your tnsnames.ora file!
The (HS=OK) parameter must be outside the SID section and specifies that this 
connector uses the Oracle Heterogeneous Service Option.

After adding the tnsnames alias and restarting the listener, a 
connectivity check is to use tnsping <alias>. 
tnsping hsodbc
should come back with a successfull message.

3) init.ora of the gateway:
The SID to use HS functionality is called in this example hsodbc. There are 
some restrictions how to name the SID (described in the Net Administrators 
Guide in detail).
At this place only a short note: don't use dots in the SID and keep it short!

The SID is also relevant for the init.ora file of the gateway. The name of the 
file is init<SID>.ora.  In this example it is called inithsodbc.ora.
The file is located at $ORACLE_HOME/hs/admin.
It should contain the following entries:
#
# HS init parameters
#
HS_FDS_CONNECT_INFO = mssql
HS_FDS_TRACE_LEVEL = off
HS_FDS_SHAREABLE_NAME=/u05/odbc/lib/libodbc.sl
#
# ODBC specific environment variables
#
set ODBCINI=/u05/odbc/odbc.ini
#
# Environment variables required for the non-Oracle system
#
#set <envvar>=<value>


Short explanation of the parameters:
HS_FDS_CONNECT_INFO points to the ODBC DSN configured in PART I of this note.
HS_FDS_SHAREABLE_NAME points to the ODBC Driver Manager library at 
$ODBC_HOME/lib/<ODBC_Driver_MANAGER_LIB>.
For the Data Direct Technologies odbc driver the generic odbc library is
called libodbc.sl. This library checks the ODBC DSN configuration and loads the
driver to the foreign database server. The name of this library may differ
from odbc vendor to vendor. Please check out the driver documentation to
figure out the generic odbc library. Also some ODBC driver vendors do not
require an ODBC Driver Manager; then the ODBC driver library itself can be
specified here. To determine if an ODBC Driver Manager is required, please 
contact the ODBC driver vendor.
(As not each ODBC Driver vendor documents its ODBC Driver Manager library and
the library name might differ from Driver Manager to Driver Manager a possible 
way to figure out the Driver Manager library name could be to check for
the existence of SQLAllocConnect ODBC function within this library:
     strings <library name> |grep -i sqlalloc
)


The set ODBCINI=/u05/odbc/odbc.ini points to the location of an odbc.ini
file you want to use with this hsodbc configuration.


4) Configuring the environment:
Normally there is nothing to configure anymore. But to test the odbc 
connectivity for the Oracle user the following should be performed:
Set the ODBCINI and ODBC_HOME environment variable and add the 
$ODBC_HOME/lib directory to the SHLIB_PATH.
(The details how to do it are described in Part I.)

Now execute as the ORACLE User (who starts the listener) 
the demoodbc program:
$ODBC_HOME/demo/demoodbc -uid sa -pwd sa mssql
A similar output should be generated:
DataDirect Technologies, Inc. ODBC Sample Application.
will connect to data source 'mssql' as user 'sa/sa'.
 
First Name      Last Name       Hire Date            Salary          Dept
----------      ---------       ---------            ------          ----
Tyler           Bennett         1977-01-06 00:00:00.000 32000.0         D101
George          Woltman         1982-07-08 00:00:00.000 53500.0         D101
Rich            Holcomb         1983-01-06 00:00:00.000 49500.0         D202
Richard         Potter          1986-12-04 00:00:00.000 15900.0         D101
David           Motsinger       1985-05-05 00:00:00.000 19250.0         D202
Tim             Sampair         1987-02-12 00:00:00.000 27000.0         D101
SQLFetch returns: SQL_NO_DATA_FOUND


5)Configuring the Oracle database
The only thing that must be done here is to create a database link:
connect with the username/password that has sufficient rights to create a 
database link (i.e. system).
The syntax is:
create [public] database link <name> 
  connect to <UID> identified by <pwd> using '<tnsalias>';

In other words, to connect to the MS SQL Server configured in the last steps, 
the syntax must be:
CREATE DATABASE LINK sqlserver
 CONNECT TO "sa" IDENTIFIED BY "sa" USING 'hsodbc';

The db link name is sqlserver. Username and password must be in double quotes,
because the username and password are case sensitive. 'hsodbc' points to 
the alias in the tnsnames.ora file that calls the HS subsystem.

If everything is configured well, a select of the EMP table -created for the
demoodbc program- should be successful:

select * from "EMP"@sqlserver;
...

(Side note: The EMP table at the MS SQL Server is in capital letters. Because
the MS SQL Server is case sensitive the EMP table must be surrounded 
by double quotes). @sqlserver points to the name of the database link to the 
MS SQL Server.
 


Part IV How to set it up for Oracle 8i on HP-UX
-----------------------------------------------

Because ODBC is independent from the Oracle version being used,
Part I of this note (setting up odbc driver) is the same for 
8i and 9i Oracle database releases.

Part III is almost the same:
The description of the tnsnames.ora and init.ora of the HS subsystem is also 
valid for 8i. Also the create database statement...
Only the listener.ora differs a little bit in the ENVS setting:
In release 9i the 32 bit libraries are stored at:
$ORACLE_HOME/lib32
in 8i the 32 bit libraries are stored in:
$ORACLE_HOME/lib
!! lib instead of lib32 !!
and the Oracle_Home is the home directory of 8i release.
So the SID section looks like:
    (SID_DESC =
      (SID_NAME = hsodbc)
      (ORACLE_HOME = /u01/app/oracle/product/8.1.7)
      (PROGRAM = hsodbc)
      (ENVS=SHLIB_PATH=/u05/odbc/lib:/u01/app/oracle/product/8.1.7/lib)
    )    

The conclusion about the directory difference is, that in 8i the environment
SHLIB_PATH misses the 32 appendix:
(export SHLIB_PATH=$ODBC_HOME/lib:$SHLIB_PATH
  echo $SHLIB_PATH
  SHLIB_PATH=/u05/odbc/lib:/u01/app/oracle/product/8.1.7/lib)


Relinking as described in Part II is completely different:
Check out for the /opt/aCC/lib/cpprt0_stub.o file again.
If it does not exist, create it as described in Part II above.

Copy this file to $ORACLE_HOME/rdbms/lib directory.
Change to $ORACLE_HOME/rdbms/lib directory.

To relink the HS executable use the following command. 
Copy/paste the command into a script on the HP-UX machine, make
it executable (chmod +x) and run it.

--- cut here ---
cc -Wl,+s -Wl,+n                               \
        -o $ORACLE_HOME/rdbms/lib/hsodbc       \
        -L $ORACLE_HOME/rdbms/lib              \
        -L $ORACLE_HOME/lib                    \
        -o $ORACLE_HOME/rdbms/lib/hsodbc       \
        $ORACLE_HOME/hs/lib/hsodbc.o           \
        $ORACLE_HOME/rdbms/lib/defopt.o        \
        $ORACLE_HOME/rdbms/lib/homts.o         \
        $ORACLE_HOME/rdbms/lib/ssdbaed.o       \
        $ORACLE_HOME/rdbms/lib/cpprt0_stub.o   \
        -L $ORACLE_HOME/hs/lib/                \
        -lnavhoa                               \
        -lnavshr                               \
        -lnvbaseshr                            \
        -lagtsh                                \
        -lpls8                                 \
        -lplp8                                 \
        -lclntsh                               \
        -lnls8                                 \
        -lcore8                                \
        -lnls8                                 \
        -lcore8                                \
        -lnls8                                 \
        `cat $ORACLE_HOME/lib/sysliblist`      \
        -lm                                    \
        -lstd -lstream -lCsup -lcl -ldld
--- cut here ---

The new hsodbc executable is built in the $ORACLE_HOME/rdbms/lib directory.
Rename the old hsodbc library $ORACLE_HOME/bin/hsodbc and then move
the new hsodbc to the $ORACLE_HOME/bin location.
Test again while entering hsodbc and pressing enter key.
A message like the following should appear:
ora817 @ /u01/app/oracle/product/8.1.7/rdbms/lib hsodbc
 
 
Oracle Corporation --- TUESDAY   OCT 15 2002 16:35:03.439
 
Heterogeneous Agent based on the following module(s):
   - External Procedure Module
   - Transaction Module
   - SQL/PLSQL Module
 
A select in SQL*Plus via the db link should be possible now.


Part V: Comments
----------------
Please be aware, that after each patch set install, the RELINK process must
be executed again manually.
Repeat the steps described in Part II after a successfull patch installation.
Else the following error occures:
SQL> select * from all_catalog@hsodbc;
select * from all_catalog@hsodbc
                          *
ERROR at line 1:
ORA-28500: connection from ORACLE to a non-Oracle system returned this message:
[Generic Connectivity Using ODBC]Bad file number; at FIND_IMAGE_SYMBOL
[C079] Failed to load dynamic library '/u05/odbc/lib/libodbc.sl'
ORA-02063: preceding 3 lines from HSODBC


RELATED DOCUMENTS
-----------------

NOTE:150458.1, Note:109621.1

Related

Products

Oracle Database Products > Oracle Database > Oracle Database > Oracle Server - Enterprise Edition

Oracle Transparent Gateway for ODBC

Keywords

HETEROGENEOUS; HP-UX; HSODBC

How To Setup Generic Connectivity - HSODBC - on Linux [ID 264567.1]

2010-02-16T11:44:00.002+07:00



		Modified 23-JUN-2008 Type FAQ Status PUBLISHED


Checked for relevance on 17-Oct-2007


PURPOSE
-------

Starting with database release 10g Oracle provides generic connectivity for
Linux.

PLEASE PAY ATTENTION: CURRENTLY (UP TO 10gR2) HSODBC IS ONLY PORTED TO
LINUX X86; NOT TO Linux X86-64bit.


SCOPE & APPLICATION
-------------------
Below is a description of setting up HSODBC (generic connectivity) for
Linux using Oracle 10g release.


How to Setup HSODBC (generic connectivity) on LINUX
---------------------------------------------------

This note is divided into two different parts:
Part I describes how to set up the ODBC driver
Part II describes the configuration process of HSODBC



Part I: Setting up the ODBC driver
----------------------------------
There are several ODBC vendors for UNIX platforms.
Below a description with the ODBC driver from
Data Direct Technologies. They also provide a 15 day trial
license for Linux based platforms. More information is available
at the following URL:
 <<http://www.datadirect.com>>

The libraries of other ODBC vendors will vary; so please make sure
to change the libraries to the libraries of your odbc vendor.

As mentioned, Data Direct Technologies provides a trial version.
If you download this trial version, please make sure that you download
also the service pack for the odbc driver -if it exists-.

Another feature of this driver is, that it also contains
some mechanism to check the ODBC connectivity.

Install the ODBC driver into a separate home directory. Let's call this
home directory ODBC_HOME (i.e. /home/odbc/dd) directory.
To install the driver, you can create a new user called odbc who owns
the software.
Install the ODBC driver as mentioned in the documentation.

- Configuring the odbc.ini file:
 The odbc.ini file is similar to an address book for the odbc driver.
 It is located by default in the ODBC_HOME directory, but can be
 placed anywhere you like it.

 A side note how odbc works: The odbc driver (nothing else than a library)
 gets a request to connect to a server described in the odbc.ini file.
 The alias for the description of the server is called:
  Data Source Name (=DSN).
 Then the driver reads the information from the odbc.ini file according to
 the specified DSN and connects to the server.
 
 The prerequisite for the odbc driver to connect to the server is the
 configured odbc.ini.
 It is divided into 3 different sections:
 [ODBC Data Sources]
 [<DSN>]
 [ODBC]

 [ODBC Data Source] is the section that contains all the available DSNs.
 [<DSN>] contains the different names of the DSNs and specifies the connect
         details.
 [ODBC] is the general section for the odbc driver

 A very simple file of the odbc.ini file may look like:
  [ODBC Data Sources]
  mssql=MS SQL Server

  [mssql]
  Driver=/home/odbc/dd/lib/ivmsss22.so
  Description=DataDirect 5.2 SQL Server Wire Protocol
  Database=<SQL Server Database>
  LogonID=<UID like:sa>
  Password=<password for the user>
  Address=<hostname of the SQL Server>,<port; default is 1433>
  QuotedId=No
  AnsiNPW=No

  [ODBC]
  IANAAppCodePage=4
  InstallDir=/home/odbc/dd
  Trace=0
  TraceDll=/home/odbc/dd/lib/odbctrac.so
  TraceFile=odbctrace.out
  UseCursorLib=0
  UseCursorLib=0

  The section [ODBC Data Sources] contains one datasource called mssql.
  The configuration behind the data source mssql is found in the section
  [mssql].
  It contains the address (and port) of the server to contact, the driver
  that should be used while connecting to the remote server and the
  user id and password of the remote server.
  The [ODBC] section contains general paremeters like tracing
  (Trace=1 enables tracing, Trace=0 disables it).
 
After configuring the odbc.ini file, the first step is to check if the
ODBC configuration works. Data Direct Technologies provides a demo program to
test the connectivity and fetches some data from the remote server.
The directory $ODBC_HOME/demo contains some sql scripts to create on the foreign
database a demo table called EMP.
For the Microsoft SQL Server for example the script is called empsqlsrv.sql.
Run this script on the SQL Server to create the EMP table. (Please make sure,
that you don't overwrite/delete any tables you need).

Before calling the program demoodbc, you need to set two environment variables:
a) LD_LIBRARY_PATH must contain the odbc library path
         export LD_LIBRARY_PATH=$ODBC_HOME/lib:$LD_LIBRARY_PATH
b) ODBCINI
         export ODBCINI=$ODBC_HOME/odbc.ini
The ODBCINI parameter will guarantee, that the newly configured
odbc.ini file from above is used.

Now calling the odbc demo program to query the remote MS SQL Server:
demoodbc -uid <user of the MS SQL Server> -pwd <appropriated password> <DSN>
like
demoodbc -uid sa -pwd sa mssql
should connect to the server and query the EMP table.

Please make sure that you can successfully query the table AND don't proceed
if this configuration fails!
If you have problems configuring the odbc driver, please contact the vendor
of the driver.



Part II: How to configure HSODBC
---------------------------------
(This section assumes that everything is done in the Oracle user account
that starts the listener!)

In general the following things must be configured:
1) listener
2) tnsnames
3) init<SID>.ora of the hs subsystem
4) environment
5) Oracle database


1) The listener needs a new SID entry like the following:
  (SID_DESC =
     (SID_NAME = hsodbc)
     (ORACLE_HOME = /home/oracle/server/10.2.0/)
     (PROGRAM = hsodbc)
     (ENVS=LD_LIBRARY_PATH=/home/oracle/server/10.2.0/lib:/home/odbc/dd/lib)
   )

Please correct the ORACLE_HOME entry and the ENVS entry according to your
installation.
If the odbc driver requires the foreign data store
client libraries (like the Progress ODBC driver), the LD_LIBRARY_PATH must
contain this library path as well:
...
 (ENVS=LD_LIBRARY_PATH=/home/oracle/server/10.2.0/lib:/home/odbc/dd/lib:
                       /progress/dcl/lib)
...

ORACLE_HOME must point to your ORACLE_HOME directory and the ENVS
string contains entries for the LD_LIBRARY_PATH.
The minimum of the LD_LIBRARY_PATH setting must contain the Oracle library
and the odbc library path; both 32 bit.
HSODBC is a 32 bit libray and thus it needs a 32 bit ODBC driver.


A correct setting of the path can be verified by typing

hsodbc

then pressing <ENTER> at the console. If the LD_LIBRARY_PATH contains
the correct libraries, the version number of HSODBC should be displayed.

So a listener.ora file can look like:
SID_LIST_LISTENER =
 (SID_LIST =
  (SID_DESC =
     (SID_NAME = hsodbc)
     (ORACLE_HOME = /home/oracle/server/10.2.0/)
     (PROGRAM = hsodbc)
     (ENVS=LD_LIBRARY_PATH=/home/oracle/server/10.2.0/lib:/home/odbc/dd/lib)
   )
 )

LISTENER =
 (DESCRIPTION_LIST =
   (DESCRIPTION =
     (ADDRESS_LIST =
       (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname of the Oracle Server>)
        (PORT = 1921))
     )
   )
 ) 


The listener must be restarted (use stop and start) after changing the
listener.ora file!


2) The tnsnames.ora needs an entry for the HSODBC alias:
HSODBC.DE.ORACLE.COM =
 (DESCRIPTION =
   (ADDRESS_LIST =
     (ADDRESS = (PROTOCOL = tcp)(HOST = <hostname of the Oracle Server)(PORT = 1521))
   )
   (CONNECT_DATA =
     (SID = hsodbc)
   )
   (HS=OK)
 )

The domain of the tns alias can differ from the one used above (de.oracle.com),
depending on the parameter in the sqlnet.ora:
NAMES.DEFAULT_DOMAIN = de.oracle.com

But the important entry is the (HS=)or (HS=OK) key word. This key word must
be added manually and opening the Net Configuration Assistants will
remove this entries from your tnsnames.ora file!
The (HS=OK) parameter must be outside the SID section and specifies that this
connector uses the Oracle Heterogeneous Service Option.

After adding the tnsnames alias and restarting the listener, a
connectivity check is to use tnsping <alias>.
tnsping hsodbc
should come back with a successfull message.

3) init.ora of the gateway:
The SID to use HS functionality is called in this example hsodbc. There are
some restrictions how to name the SID (described in the Net Administrators
Guide in detail).
At this place only a short note: don't use dots in the SID and keep it short!

The SID is also relevant for the init.ora file of the gateway. The name of the
file is init<SID>.ora.  In this example it is called inithsodbc.ora.
The file is located at $ORACLE_HOME/hs/admin.
It should contain the following entries:
#
# HS init parameters
#
HS_FDS_CONNECT_INFO = mssql
HS_FDS_TRACE_LEVEL = off
HS_FDS_SHAREABLE_NAME = /home/odbc/dd/lib/libodbc.so
#
# ODBC specific environment variables
#
set ODBCINI=/home/odbc/dd/odbc.ini
#
# Environment variables required for the non-Oracle system
#
#set <envvar>=<value>


Short explanation of the parameters:
HS_FDS_CONNECT_INFO points to the ODBC DSN configured in PART I of this note.
HS_FDS_SHAREABLE_NAME points to the ODBC Driver Manager library at
$ODBC_HOME/lib/<ODBC_Driver_MANAGER_LIB>.
For the Data Direct Technologies odbc driver the generic odbc library on Linux is
called libodbc.so. This library checks the ODBC DSN configuration and loads the
driver to the foreign database server. The name of this library may differ
from odbc vendor to vendor. Please check out the driver documentation to
figure out the generic odbc library. Also some ODBC driver vendors do not
require an ODBC Driver Manager; then the ODBC driver library itself can be
specified here. To determine if an ODBC Driver Manager is required, please
contact the ODBC driver vendor.
(As not each ODBC Driver vendor documents its ODBC Driver Manager library and
the library name might differ from Driver Manager to Driver Manager a possible
way to figure out the Driver Manager library name could be to check for
the existence of SQLAllocConnect ODBC function within this library:
    strings <library name> |grep -i sqlalloc
)

The set ODBCINI=/home/odbc/dd/odbc.ini points to the location of an odbc.ini
file you want to use with this hsodbc configuration.


4) Configuring the environment:
Normally there is nothing to configure anymore. But to test the odbc
connectivity for the Oracle user the following should be performed:
Set the ODBCINI and ODBC_HOME environment variable and add the
$ODBC_HOME/lib directory to the $LD_LIBRARY_PATH.
(The details how to do it are described in Part I.)

Now execute as the ORACLE User (who starts the listener)
the demoodbc program:
$ODBC_HOME/demo/demoodbc -uid sa -pwd sa mssql
A similar output should be generated:
DataDirect Technologies, Inc. ODBC Sample Application.
will connect to data source 'mssql' as user 'sa/sa'.

First Name      Last Name       Hire Date            Salary          Dept
----------      ---------       ---------            ------          ----
Tyler           Bennett         1977-01-06 00:00:00.000 32000.0         D101
George          Woltman         1982-07-08 00:00:00.000 53500.0         D101
Rich            Holcomb         1983-01-06 00:00:00.000 49500.0         D202
Richard         Potter          1986-12-04 00:00:00.000 15900.0         D101
David           Motsinger       1985-05-05 00:00:00.000 19250.0         D202
Tim             Sampair         1987-02-12 00:00:00.000 27000.0         D101
SQLFetch returns: SQL_NO_DATA_FOUND


5)Configuring the Oracle database
The only thing that must be done here is to create a database link:
connect with the username/password that has sufficient rights to create a
database link (i.e. system).
The syntax is:
create [public] database link <name>
 connect to <UID> identified by <pwd> using '<tnsalias>';

In other words, to connect to the MS SQL Server configured in the last steps,
the syntax must be:
CREATE DATABASE LINK sqlserver
CONNECT TO "sa" IDENTIFIED BY "sa" USING 'hsodbc';

The db link name is sqlserver. Username and password must be in double quotes,
because the username and password are case sensitive. 'hsodbc' points to
the alias in the tnsnames.ora file that calls the HS subsystem.

If everything is configured well, a select of the EMP table -created for the
demoodbc program- should be successful:

select * from "EMP"@sqlserver;
...

(Side note: The EMP table at the MS SQL Server is in capital letters. Because
the MS SQL Server is case sensitive the EMP table must be surrounded
by double quotes). @sqlserver points to the name of the database link to the
MS SQL Server.

RELATED DOCUMENTS
-----------------

The relevant manuals are :-
Heterogeneous Connectivity Administrator’s Guide 10g Release 2 (10.2)

Note:261726.1 -  Generic Connectivity Available on Linux with 10g

Related

Products

Oracle Database Products > Oracle Database > Oracle Database > Oracle Server - Enterprise Edition

Keywords

CONFIGURATION; CONNECTIVITY; HSODBC; ODBC

Tuning Generic Connectivity And Gateways [ID 230543.1]

2010-02-16T11:20:00.002+07:00



		Modified 26-OCT-2009 Type BULLETIN Status PUBLISHED


Checked for relevance on 26-Oct-2009
Checked for relevance on 20-Feb-2008
Checked for relevance on 08-Aug-2006

PURPOSE
-------

General tips tuning HS/Gateways


SCOPE & APPLICATION
-------------------

This bulletin will give some hints how to tune
generic connectivity or open gateways.


Tips tuning generic connectivity/gateways
-----------------------------------------

There are no real parameters to tune generic connectivity or the
open gateways. Everything in the following article is to demonstrate
the behaviour of heterogeneous services, to explain the working mechanism and
what can be done to improve the performance.

Background information: HS connects from an Oracle database to a foreign
data source and fetches records. The amount of the fetched records
reflects the time needed to transfer them.


The simplest way to improve performance is to reduce the amount
of fetched data.

But how to achive this?

- RULE 1:
         AVOID select * from remote table

That's a very important statement!

But fetching all records from the remote database can sometimes
occur without wanting to fetch all the records. The cause is
called POST PROCESSING:
The Transparent Gateways tell the Oracle database during connect time
which functions and operators they support. If the remote database does
not support this function or operator, then the Oracle database must
execute the function or operand. But to do this ALL records from
the remote database must be fetched and processed locally in the Oracle
database.

- RULE 2:
        pay attention to which functions/operands are supported by HS.
        Unsupported functions are post processed!

How to determine what's going on and how many records are fetched:
The IBM gateways support explain plan. With this feature you're able
to see the statements passed and processed.
But this feature is not yet implemented in the generic connectivity
or open gateways.
So the only way to figure out what statements are passed between the
Oracle database and the remote database is to verify the HS traces.



One option to solve this issue is to CREATE VIEWS.

   With access to the remote database:
       it is possible to create a view at the remote database that
       pre selects the records.
       HS now performs its own query only to this view and processes only
       the pre selected records.
   With no access to the remote database:
       the whole operation could be used in conjunction with
       DBMS_HS_PASSTHROUGH packages. 
       DBMS_HS_PASSTHROUGH allows the creation of views at the remote database
       as well as the sending of select statements as they are
       to the foreign database without being pre prosssed.

Another option is to use the COST-BASED optimizer.
The cost-based optimizer uses indexes on remote tables and considers
more execution plans then the rule based optimizer.


The real performance tuning is to reduce the amount of data fetched by
the HS agent.


But sometimes adapting the interface might improve performance as well.
By default, an agent fetches data from the non-Oracle system until
it has enough data retrieved to send back to the HS related part in the
Oracle database. The agent reblocks the data between the agent and the
Oracle database server in sizes defined by the value of HS_RPC_FETCH_SIZE.

The other part is the transfer of the foreign database related interface
like odbc and the agent.
This transfer can be manipulated by setting the parameter
              HS_FDS_FETCH_ROWS.
The default value is 20, but it makes sense to increase this value by
adapting it to the fetched rows of the remote database.
While some ODBC drivers initialize the SQL_FETCH_ROWS to 100 the
HS_FDS_FETCH_ROWS should be set to 100 or even to n*SQL_FETCH_ROWS.

For example, assume that you set HS_RPC_FETCH_SIZE to 64K and HS_FDS_FETCH_ROWS
to 100 rows. Assume that each row is approximately 600 bytes in size,
so that the 100 rows are approximately 60K.
The agent starts fetching 100 rows from the non-Oracle system.
Because there is only 60K bytes of data in the agent, the agent does not
send the data to the Oracle database server. Instead, the agent
fetches the next 100 rows from the non-Oracle system.
Now the agent is filled with 120K of data and the first 64K can be sent
to the Oracle database server.
There is 56K of data left in the agent. The agent fetches another 100 rows
from the non-Oracle system before sending the next 64K of data to the Oracle
database server....

From the description above it looks like the agent is a bottle neck. It only
sends data after a buffer is filled. So a better idea would be to STREAM the
data.
If the hs agent supports array fetching (please check out the documentation for
a specific type of agent) the blocking can be switched of. Set the
initialization parameter
                   HS_RPC_FETCH_REBLOCKING
to OFF.

According to the sample from above this means that the first 100 rows are
immediately sent to the Oracle server.

So in theory a performance improvement will take place by setting
HS_RPC_FETCH_REBLOCKING to OFF and HS_FDS_FETCH_ROWS to a value of
n*SQL_FETCH_ROWS.

KEEP IN MIND:
All these parameters manipulate the DATA TRANSFER. While each fetched
record depends on networking capacity, on CPU usage..., the best tuning
mechanism is still to reduce the amount of transferred data.

And this can be achieved by CREATING VIEWS to preselect
the fetched records as described at the beginning of this document.



RELATED DOCUMENTS
-----------------
Note.234996.1

Related

Products

Oracle Database Products > Oracle Database > Oracle Database > Oracle Server - Enterprise Edition

SQL Server 2005 Books Online (November 2008) DBCC CHECKDB (Transact-SQL)

2010-02-15T16:56:00.001+07:00

Updated: 17 November 2008

Checks the logical and physical integrity of all the objects in the specified database by performing the following operations:

Runs DBCC CHECKALLOC [ http://msdn.microsoft.com/en-us/library/ms188422(SQL.90).aspx ] on the database.
Runs DBCC CHECKTABLE [ http://msdn.microsoft.com/en-us/library/ms174338(SQL.90).aspx ] on every table and view in the database.
Runs DBCC CHECKCATALOG [ http://msdn.microsoft.com/en-us/library/ms186720(SQL.90).aspx ] on the database.
Validates the contents of every indexed view in the database.
Validates the Service Broker data in the database.

This means that the DBCC CHECKALLOC, DBCC CHECKTABLE, or DBCC CHECKCATALOG commands do not have to be run separately from DBCC CHECKDB. For more detailed information about the checks that these commands perform, see the descriptions of these commands.

Transact-SQL Syntax Conventions [ http://msdn.microsoft.com/en-us/library/ms177563(SQL.90).aspx ]

Syntax

DBCC CHECKDB 
[     [ ( database_name | database_id | 0         [ , NOINDEX         | , { REPAIR_ALLOW_DATA_LOSS | REPAIR_FAST | REPAIR_REBUILD } ]         ) ]     [ WITH         {             [ ALL_ERRORMSGS ]             [ , NO_INFOMSGS ]             [ , TABLOCK ]             [ , ESTIMATEONLY ]             [ , { PHYSICAL_ONLY | DATA_PURITY } ]         }     ]
]

Arguments

database_name | database_id | 0: Is the name or ID of the database for which to run integrity checks. If not specified, or if 0 is specified, the current database is used. Database names must comply with the rules for identifiers [ http://msdn.microsoft.com/en-us/library/ms175874(SQL.90).aspx ] .

NOINDEX: Specifies that intensive checks of nonclustered indexes for user tables should not be performed. This decreases the overall execution time. NOINDEX does not affect system tables because integrity checks are always performed on system table indexes.

REPAIR_ALLOW_DATA_LOSS | REPAIR_FAST | REPAIR_REBUILD

Specifies that DBCC CHECKDB repair the found errors. The specified database must be in single-user mode to use one of the following repair options.

REPAIR_ALLOW_DATA_LOSS: Tries to repair all reported errors. These repairs can cause some data loss.

REPAIR_FAST: Maintains syntax for backward compatibility only. No repair actions are performed.

REPAIR_REBUILD: Performs both minor, quick repairs, such as repairing extra keys in nonclustered indexes, and time-consuming repairs such as rebuilding indexes. These repairs can be performed without risk of data loss.

Important:
Use the REPAIR options only as a last resort. To repair errors, we recommend restoring from a backup. Repair operations do not consider any of the constraints that may exist on or between tables. If the specified table is involved in one or more constraints, we recommend running DBCC CHECKCONSTRAINTS after a repair operation. If you must use REPAIR, run DBCC CHECKDB without a repair option to find the repair level to use. If you use the REPAIR_ALLOW_DATA_LOSS level, we recommend that you back up the database before you run DBCC CHECKDB with this option.

Important:

ALL_ERRORMSGS

Displays all reported errors per object. In SQL Server 2005 Service Pack 3 (SP3), all error messages are displayed by default. Specifying or omitting this option has no effect. In earlier versions of SQL Server, only the first 200 error messages for each object are displayed if ALL_ERRORMSGS is not specified. Error messages are sorted by object ID, except for those messages generated from tempdb database [ http://msdn.microsoft.com/en-us/library/ms190768(SQL.90).aspx ] .

In SQL Server Management Studio, the maximum number of error messages returned is 1000. Using Management Studio, you may need to run DBCC CHECKDB multiple times to get a complete list of errors when ALL_ERRORMSGS is specified. We recommend that you run the DBCC command by using the sqlcmd utility [ http://msdn.microsoft.com/en-us/library/ms162773(SQL.90).aspx ] or by scheduling a SQL Server Agent job to run the command and direct the output to a file. Either of these methods will ensure that running the command once will report all error messages.

NO_INFOMSGS: Suppresses all informational messages.

TABLOCK

Causes DBCC CHECKDB to obtain locks instead of using an internal database snapshot. This includes a short-term exclusive (X) lock on the database. TABLOCK will cause DBCC CHECKDB to run faster on a database under heavy load, but decreases the concurrency available on the database while DBCC CHECKDB is running. For more information about locks, see Lock Modes [ http://msdn.microsoft.com/en-us/library/ms175519(SQL.90).aspx ] .

TABLOCK limits the checks that are performed; DBCC CHECKCATALOG is not run on the database, and Service Broker data is not validated.

ESTIMATEONLY: Displays the estimated amount of tempdb space that is required to run DBCC CHECKDB with all the other specified options. The actual database check is not performed.

PHYSICAL_ONLY: Limits the checking to the integrity of the physical structure of the page and record headers, the physical structure of B-trees, and the allocation consistency of the database. This check is designed to provide a small overhead check of the physical consistency of the database, but it can also detect torn pages, checksum failures, and common hardware failures that can compromise a user's data. PHYSICAL_ONLY always implies NO_INFOMSGS and is not allowed with any one of the repair options.

DATA_PURITY

Causes DBCC CHECKDB to check the database for column values that are not valid or out-of-range. For example, DBCC CHECKDB detects columns with date and time values that are larger than or less than the acceptable range for the datetime data type; or decimal or approximate-numeric data type columns with scale or precision values that are not valid.

For databases created in SQL Server 2005, column-value integrity checks are enabled by default and do not require the DATA_PURITY option. For databases upgraded from earlier versions of SQL Server, column-value checks are not enabled by default until DBCC CHECKDB WITH DATA_PURITY has been run error free on the database. After this, DBCC CHECKDB checks column-value integrity by default. For more information about how CHECKDB might be affected by upgrading database from earlier versions of SQL Server, see the Remarks section later in this topic.

If PHYSICAL_ONLY is specified, column-integrity checks are not performed.

Validation errors reported by this option cannot be fixed by using DBCC repair options. For information about manually correcting these errors, see Knowledge Base article 923247: Troubleshooting DBCC error 2570 in SQL Server 2005 [ http://support.microsoft.com/kb/923247.aspx ] .

Result Sets

DBCC CHECKDB returns the following result set. The values might vary except when the ESTIMATEONLY, PHYSICAL_ONLY, or NO_INFOMSGS options are specified:

DBCC results for 'model'.
Service Broker Msg 9675, Level 10, State 1: Message Types analyzed: 13.
Service Broker Msg 9676, Level 10, State 1: Service Contracts analyzed: 5.
Service Broker Msg 9667, Level 10, State 1: Services analyzed: 3.
Service Broker Msg 9668, Level 10, State 1: Service Queues analyzed: 3.
Service Broker Msg 9669, Level 10, State 1: Conversation Endpoints analyzed: 0.
Service Broker Msg 9674, Level 10, State 1: Conversation Groups analyzed: 0.
Service Broker Msg 9670, Level 10, State 1: Remote Service Bindings analyzed: 0.
DBCC results for 'sys.sysrowsetcolumns'.
There are 630 rows in 7 pages for object 'sys.sysrowsetcolumns'.
DBCC results for 'sys.sysrowsets'.
There are 97 rows in 1 pages for object 'sys.sysrowsets'.
DBCC results for 'sysallocunits'.
There are 195 rows in 3 pages for object 'sysallocunits'.
There are 0 rows in 0 pages for object "sys.sysasymkeys".
DBCC results for 'sys.syssqlguides'.
There are 0 rows in 0 pages for object "sys.syssqlguides".
DBCC results for 'sys.queue_messages_1977058079'.
There are 0 rows in 0 pages for object "sys.queue_messages_1977058079".
DBCC results for 'sys.queue_messages_2009058193'.
There are 0 rows in 0 pages for object "sys.queue_messages_2009058193".
DBCC results for 'sys.queue_messages_2041058307'.
There are 0 rows in 0 pages for object "sys.queue_messages_2041058307".
CHECKDB found 0 allocation errors and 0 consistency errors in database 'model'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

DBCC CHECKDB returns the following result set (message) when NO_INFOMSGS is specified:

The command(s) completed successfully.

DBCC CHECKDB returns the following result set when PHYSICAL_ONLY is specified:

DBCC results for 'model'.
CHECKDB found 0 allocation errors and 0 consistency errors in database 'master'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

DBCC CHECKDB returns the following result set when ESTIMATEONLY is specified.

Estimated TEMPDB space needed for CHECKALLOC (KB) 
------------------------------------------------- 
13

(1 row(s) affected)

Estimated TEMPDB space needed for CHECKTABLES (KB) 
-------------------------------------------------- 
57

(1 row(s) affected)

DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Remarks

In earlier versions of SQL Server, the values for the per-table and per-index row count and page counts can become incorrect. Under certain circumstances, one or more of these values might even become negative. In SQL Server 2005, these values are always maintained correctly. Therefore, databases that are created on SQL Server 2005 should never contain incorrect counts; however, databases that are upgraded to SQL Server 2005 might. This is not a corruption of any data stored in the database. DBCC CHECKDB has been enhanced to detect when any one of these counts becomes negative. When negative counts are detected, the DBCC CHECKDB output contains a warning and a recommendation to run DBCC UPDATEUSAGE [ http://msdn.microsoft.com/en-us/library/ms188414(SQL.90).aspx ] to correct the issue. Although it may appear as if upgrading the database to SQL Server 2005 has caused this issue, the incorrect counts were present before the upgrade procedure.

DBCC CHECKDB does not examine disabled indexes. For more information about disabled indexes, see Disabling Indexes [ http://msdn.microsoft.com/en-us/library/ms177406(SQL.90).aspx ] .

If a user-defined type is marked as being byte ordered, there must only be one serialization of the user-defined type. Not having a consistent serialization of byte-ordered user-defined types causes error 2537 when DBCC CHECKDB is run. For more information, see User-Defined Type Requirements [ http://msdn.microsoft.com/en-us/library/ms131082(SQL.90).aspx ] .

Because the Resource database [ http://msdn.microsoft.com/en-us/library/ms190940(SQL.90).aspx ] is modifiable only in single-user mode, the DBCC CHECKDB command cannot be run on it directly. However, when DBCC CHECKDB is executed against the master database [ http://msdn.microsoft.com/en-us/library/ms187837(SQL.90).aspx ] , a second CHECKDB is also run internally on the Resource database. This means that DBCC CHECKDB can return extra results. The command returns extra result sets when no options are set, or when either the PHYSICAL_ONLY or ESTIMATEONLY option is set.

In versions of SQL Server 2005 before SP2, executing DBCC CHECKDB clears the plan cache for the instance of SQL Server. Clearing the plan cache causes recompilation of all later execution plans and may cause a sudden, temporary decrease in query performance. In SP2, executing DBCC CHECKDB does not clear the plan cache.

Internal Database Snapshot

DBCC CHECKDB uses an internal database snapshot for the transactional consistency needed to perform these checks. This prevents blocking and concurrency problems when these commands are executed. For more information, see Understanding Sparse File Sizes in Database Snapshots [ http://msdn.microsoft.com/en-us/library/ms175823(SQL.90).aspx ] and the DBCC Internal Database Snapshot Usage section in DBCC (Transact-SQL) [ http://msdn.microsoft.com/en-us/library/ms188796(SQL.90).aspx ] . If a snapshot cannot be created, or TABLOCK is specified, DBCC CHECKDB acquires locks to obtain the required consistency. In this case, an exclusive database lock is required to perform the allocation checks, and shared table locks are required to perform the table checks.

DBCC CHECKDB fails when run against master if an internal database snapshot cannot be created.

Running DBCC CHECKDB against tempdb does not perform any allocation or catalog checks and must acquire shared table locks to perform table checks. This is because, for performance reasons, database snapshots are not available on tempdb. This means that the required transactional consistency cannot be obtained.

Best Practices

In SQL Server 2005, running DBCC CHECKDB with no options may take considerably longer than in earlier versions. This behavior occurs for the following reasons:

The logical checks introduced are more comprehensive.

Some of the underlying structures to be checked are more complex.

Many new checks have been introduced to include the new features in SQL Server 2005.

Therefore, we recommend that you use the PHYSICAL_ONLY option for frequent use on production systems. Using PHYSICAL_ONLY can greatly shorten run-time for DBCC CHECKDB on large databases. We also recommend that you periodically run DBCC CHECKDB with no options. How frequently you should perform these runs depends on individual businesses and their production environments.

Checking Objects in Parallel

By default, DBCC CHECKDB performs parallel checking of objects. The degree of parallelism is automatically determined by the query processor. The maximum degree of parallelism is configured just like parallel queries. To restrict the maximum number of processors available for DBCC checking, use sp_configure [ http://msdn.microsoft.com/en-us/library/ms188787(SQL.90).aspx ] . For more information, see max degree of parallelism Option [ http://msdn.microsoft.com/en-us/library/ms181007(SQL.90).aspx ] . Parallel checking can be disabled by using trace flag 2528. For more information, see Trace Flags (Transact-SQL) [ http://msdn.microsoft.com/en-us/library/ms188396(SQL.90).aspx ] .

Understanding DBCC Error Messages

After the DBCC CHECKDB command finishes, a message is written to the SQL Server error log. If the DBCC command successfully executes, the message indicates success and the amount of time that the command ran. If the DBCC command stops before completing the check because of an error, the message indicates that the command was terminated, a state value, and the amount of time the command ran. The following table lists and describes the state values that can be included in the message.

State	Description
0	Error number 8930 was raised. This indicates a corruption in metadata that terminated the DBCC command.
1	Error number 8967 was raised. There was an internal DBCC error.
2	A failure occurred during emergency mode database repair.
3	This indicates a corruption in metadata that terminated the DBCC command.
4	An assert or access violation was detected.
5	An unknown error occurred that terminated the DBCC command.

Error Reporting

In SQL Server 2005 Service Pack 1, a dump file (SQLDUMPnnnn.txt) is created in the SQL Server LOG directory whenever DBCC CHECKDB detects a corruption error. When the Feature Usage data collection and Error Reporting features are enabled for the instance of SQL Server, the file is automatically forwarded to Microsoft. The collected data is used to improve SQL Server functionality. For more information, see Error and Usage Report Settings [ http://msdn.microsoft.com/en-us/library/ms144283(SQL.90).aspx ] .

The dump file contains the results of the DBCC CHECKDB command and additional diagnostic output. Access is limited to the SQL Server service account and members of the sysadmin role. By default, the sysadmin role contains all members of the Windows BUILTIN\Administrators group and the local administrator's group. The DBCC command does not fail if the data collection process fails.

Resolving Errors

If any errors are reported by DBCC CHECKDB, we recommend restoring the database from the database backup instead of running REPAIR with one of the REPAIR options. If no backup exists, running repair corrects the errors reported. The repair option to use is specified at the end of the list of reported errors. However, correcting the errors by using the REPAIR_ALLOW_DATA_LOSS option might require deleting some pages, and therefore some data.

Under some circumstances, values might be entered into the database that are not valid or out-of-range based on the data type of the column. In SQL Server 2000, DBCC CHECKDB does not perform range or integrity checks on these column values. However, in SQL Server 2005, DBCC CHECKDB can detect column values that are not valid for all column data types. Therefore, running DBCC CHECKDB with the DATA_PURITY option on databases that have been upgraded from earlier versions of SQL Server might reveal preexisting column-value errors. Because SQL Server 2005 cannot automatically repair these errors, the column value must be manually updated. If CHECKDB detects such an error, CHECKDB returns a warning, the error number 2570, and information to identify the affected row and manually correct the error.

The repair can be performed under a user transaction to let the user roll back the changes that were made. If repairs are rolled back, the database will still contain errors and must be restored from a backup. After repairs are completed, back up the database.

Resolving Errors in Database Emergency Mode

When a database has been set to emergency mode by using the ALTER DATABASE [ http://msdn.microsoft.com/en-us/library/ms174269(SQL.90).aspx ] statement, DBCC CHECKDB can perform some special repairs on the database if the REPAIR_ALLOW_DATA_LOSS option is specified. These repairs may allow for ordinarily unrecoverable databases to be brought back online in a physically consistent state. These repairs should be used as a last resort and only when you cannot restore the database from a backup. When the database is set to emergency mode, the database is marked READ_ONLY, logging is disabled, and access is limited to members of the sysadmin fixed server role.

Note:
You cannot run the DBCC CHECKDB command in emergency mode inside a user transaction and roll back the transaction after execution.

When the database is in emergency mode and DBCC CHECKDB with the REPAIR_ALLOW_DATA_LOSS clause is run, the following actions are taken:

DBCC CHECKDB uses pages that have been marked inaccessible because of I/O or checksum errors, as if the errors have not occurred. Doing this increases the chances for data recovery from the database.

DBCC CHECKDB attempts to recover the database using regular log-based recovery techniques.

If, because of transaction log corruption, database recovery is unsuccessful, the transaction log is rebuilt. Rebuilding the transaction log may result in the loss of transactional consistency.

If the DBCC CHECKDB command succeeds, the database is in a physically consistent state and the database status is set to ONLINE. However, the database may contain one or more transactional inconsistencies. We recommend that you run DBCC CHECKCONSTRAINTS [ http://msdn.microsoft.com/en-us/library/ms189496(SQL.90).aspx ] to identify any business logic flaws and immediately back up the database.

If the DBCC CHECKDB command fails, the database cannot be repaired.

Running DBCC CHECKDB with REPAIR_ALLOW_DATA_LOSS in Replicated Databases

Running the DBCC CHECKDB command with the REPAIR_ALLOW_DATA_LOSS option can affect user databases (publication and subscription databases) and the distribution database used by replication. Publication and subscription databases include published tables and replication metadata tables. Be aware of the following potential issues in these databases:

Published tables. Actions performed by the CHECKDB process to repair corrupt user data might not be replicated:
- Merge replication uses triggers to track changes to published tables. If rows are inserted, updated, or deleted by the CHECKDB process, triggers do not fire; therefore, the change is not replicated.
- Transactional replication uses the transaction log to track changes to published tables. The Log Reader Agent then moves these changes to the distribution database. Some DBCC repairs, although logged, cannot be replicated by the Log Reader Agent. For example, if a data page is deallocated by the CHECKDB process, the Log Reader Agent does not translate this to a DELETE statement; therefore, the change is not replicated.

Replication metadata tables. Actions performed by the CHECKDB process to repair corrupt replication metadata tables require removing and reconfiguring replication.

If you have to run the DBCC CHECKDB command with the REPAIR_ALLOW_DATA_LOSS option on a user database or distribution database:

Quiesce the system: Stop activity on the database and at all other databases in the replication topology, and then try to synchronize all nodes. For more information, see How to: Quiesce a Replication Topology (Replication Transact-SQL Programming) [ http://msdn.microsoft.com/en-us/library/ms147385(SQL.90).aspx ] .

Execute DBCC CHECKDB.

If the DBCC CHECKDB report includes repairs for any tables in the distribution database or any replication metadata tables in a user database, remove and reconfigure replication. For more information, see Removing Replication [ http://msdn.microsoft.com/en-us/library/ms152757(SQL.90).aspx ] .

If the DBCC CHECKDB report includes repairs for any replicated tables, perform data validation to determine whether there are differences between the data in the publication and subscription databases. For more information, see Data at the Publisher and Subscriber Do Not Match [ http://msdn.microsoft.com/en-us/library/ms151272(SQL.90).aspx ] .

Permissions

Requires membership in the sysadmin fixed server role or the db_owner fixed database role.

Examples

A. Checking both the current and the AdventureWorks databases

The following example executes DBCC CHECKDB for the current database and for the AdventureWorks database.

-- Check the current database.
DBCC CHECKDB;
GO
-- Check the AdventureWorks database without nonclustered indexes.
DBCC CHECKDB (AdventureWorks, NOINDEX);
GO

B. Checking the current database, suppressing informational messages

The following example checks the current database and suppresses all informational messages.

DBCC CHECKDB WITH NO_INFOMSGS;
GO

Reference

DBCC (Transact-SQL) [ http://msdn.microsoft.com/en-us/library/ms188796(SQL.90).aspx ]

sp_helpdb (Transact-SQL) [ http://msdn.microsoft.com/en-us/library/ms178568(SQL.90).aspx ]

System Tables (Transact-SQL) [ http://msdn.microsoft.com/en-us/library/ms179932(SQL.90).aspx ]

Other Resources

Physical Database Architecture [ http://msdn.microsoft.com/en-us/library/ms179276(SQL.90).aspx ]

Understanding Sparse File Sizes in Database Snapshots [ http://msdn.microsoft.com/en-us/library/ms175823(SQL.90).aspx ]

Troubleshooting DBCC Errors on Indexed Views [ http://msdn.microsoft.com/en-us/library/ms189274(SQL.90).aspx ]

Optimizing DBCC CHECKDB Performance [ http://msdn.microsoft.com/en-us/library/ms175515(SQL.90).aspx ]

Help and Information

Getting SQL Server 2005 Assistance [ http://msdn.microsoft.com/en-us/library/ms166016(SQL.90).aspx ]

How to Configure Generic Connectivity (HSODBC) on Linux 32 bit using DB2Connect [ID 375624.1]

2010-02-15T15:42:00.001+07:00



		Modified 07-DEC-2009 Type HOWTO Status PUBLISHED

In this Document
Goal
Solution
References

Applies to:

Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 10.2.0.5 - Release: to 10.2
Linux x86
***Checked for relevance on 07-Dec-2009***

Goal

This note describes the setup of generic connectivity on Linux (32 bit) using DB2Connect and IBM ODBC driver to connect to a DB2 database.

Please be aware that HSODBC for Oracle on Linux 64bit is not available. You need to use Database Gateway for ODBC (DG4ODBC) 11g on this platform

Solution

How to Setup HSODBC (generic connectivity) on Linux using IBM ODBC driver included in DB2Connect.

This note is divided into three different parts:
Part I describes how to set up the DBConnect and IBM's ODBC driver
Part II describes the configuration process of HSODBC
Part III describes dedicated HSODBC/ DB2 ODBC errors

Part I: Setting up the ODBC driver
The odbc driver being used in this note is part of the DB2Connect package release 8.1 fixpack 12. The required ODBC Driver Manager is the driver manager from UnixODBC.

Please install the DB2Connect package according to the installation instructions from IBM.

The first step now is to configure the ODBC driver.

- Configuring the odbc.ini file:
The ODBC.INI file for the IBM ODBC driver included in DB2 Connect is very simple and requires ONLY the Driver path:

[toolsdb]
Driver = /home/db2inst1/sqllib/lib/libdb2.so

The ODBC DSN [toolsdb] refers to the database alias defined in the instance refered by by DB2INSTANCE parameter:
For example DB2INSTANCE environment variable is pointing to db2inst1, then connect as DB2 user (db2inst1) to this instance db2inst1 using utility "db2" and query the database aliases -> list database directory
db2 => list database directory
System Database Directory
Number of entries in the directory = 2
Database 1 entry:

Database alias = TOOLSDB
Database name = TOOLSDB
Node name = ZKU118
Database release level = a.00
Comment =
Directory entry type = Remote
Catalog database partition number = -1
Alternate server hostname =
Alternate server port number =

The Driver Manager unixODBC provides a test utility to test the ODBC connection to the DB2 database.
Make sure you have set DB2INSTANCE as environment variable to the DB2 database storing the network alias being refered in the ODBC.INI file.
In addition make sure environment variable ODBCINI is set and pointing to an odbc.ini file containing the db2 DSN.

Then test: isql -v <username> <password> <DSN>

Part II: How to configure HSODBC
(This section assumes that everything is done in the Oracle user account
that starts the listener!)

In general the following things must be configured:
1) listener
2) tnsnames
3) init<SID>.ora of the hs subsystem
4) environment
5) Oracle database

1) The listener needs a new SID entry like the following:
(SID_DESC =
(ORACLE_HOME = /oracle/product/10.2.0.1.0)
(SID_NAME = hsdb2udb)
(PROGRAM = hsodbc)
(ENVS=LD_LIBRARY_PATH=/oracle/product/10.2.0.1.0/lib:/home/db2inst1/sqllib/lib:/usr/lib)

Please correct the ORACLE_HOME entry and the ENVS entry according to your installation.

The LD_LIBRARY_PATH in the listener.ora contains the Oracle library directory (32 bit), the location of the ODBC Driver Manager (/usr/lib) and the DB2 library path.

The listener must be STOPPED and STARTED after changing the listener.ora! Please do not perform a restart of the listener.

2) The tnsnames.ora needs an entry for the HSODBC alias:
db2.de.oracle.com=
(DESCRIPTION =
     (ADDRESS_LIST =
       (ADDRESS = (PROTOCOL = TCP)(HOST = kgronau-pc)(PORT = 15102))
     )
    (CONNECT_DATA =
    (SERVICE_NAME = hsdb2udb)
    )
    (HS = OK)
   )

The domain of the tns alias can differ from the one used above (de.oracle.com),
depending on the parameter in the sqlnet.ora:
NAMES.DEFAULT_DOMAIN = de.oracle.com

But the important entry is the (HS=OK) key word. The (HS=OK) parameter must be outside the SID section and specifies that this connector uses the Oracle Heterogeneous Service Option.
Net Configuration Assistant/ Database Creation Assistant might remove entries like (HS=) from your tnsnames.ora file; so please make sure (HS=OK) is used!

After adding the tnsnames alias and stopping/starting the listener, a connectivity check is to use tnsping <alias>.
tnsping hsodbc
should come back with a successfull message.

3) init.ora of the gateway:
The SID to use HS functionality is called in this example hsdb2udb. There are some restrictions how to name the SID (described in the Net Administrators Guide in detail).
At this place only a short note: don't use dots in the SID and keep it short!

The SID is also relevant for the init.ora file of the gateway. The name of the
file is init<SID>.ora. In this example it is called inithsdb2udb.ora.
The file is located at $ORACLE_HOME/hs/admin.
It should contain the following entries:
#
# HS init parameters
#
#
# HS init parameters
#
HS_FDS_CONNECT_INFO = toolsdb
HS_FDS_TRACE_LEVEL = debug
HS_FDS_SHAREABLE_NAME = /usr/lib/libodbc.so

#
# ODBC specific environment variables
#
set ODBCINI=/etc/odbc.ini

#
# Environment variables required for the non-Oracle system
#
set DB2INSTANCE=db2inst1

Short explanation of the parameters:
HS_FDS_CONNECT_INFO points to the ODBC DSN configured in PART I of this note.
HS_FDS_SHAREABLE_NAME points to the ODBC Driver Manager library; in our case the unixODBC Driver Manager

The set ODBCINI=/InformixSDK/etc/odbc.ini points to the location of an odbc.ini file you want to use with this hsodbc configuration.

set DB2INSTANCE=db2inst1 points to the DB2 instance containing the database alias for the DB2 database.

4) Configuring the Oracle database
The only thing that must be done here is to create a database link:
connect with the username/password that has sufficient rights to create a
database link (i.e. system).
The syntax is:
create [public] database link <name>
connect to <UID> identified by <pwd> using '<tnsalias>';

In other words, to connect to the DB2 database the syntax must be:
CREATE DATABASE LINK db2
CONNECT TO "<valid DB2 user name>" IDENTIFIED BY "<valid password for this DB2 user>" USING 'db2.de.oracle.com';

The db link name is db2. Username and password must be in double quotes, because the username and password of the DB2 database might be case sensitive. 'db2.de.oracle.com' points to the alias in the tnsnames.ora file that calls the HS subsystem.

If everything is configured well, a select of a DB2 table or the DUAL table which is then translated by HSODBC should be successful:

select user from dual@db2;
...

Part III: Informix/HSODBC related errors
Common errors are coverd in a separate note:'
Note.234517.1 Ext/Pub How to Resolve Common Errors Encountered while using Transparent Gateways or Generic Connectivity

Problems that might occur using DB2 ODBC driver:

Problem 1:
Enabling ODBC tracing:

Solution 1:
ODBC tracing for the IBM ODBC driver is enabled by adding into the COMMON section of the db2cli.ini file:
[COMMON]
Trace=1
TraceFileName=/tmp/db2_odbc.trc

Problem 2:
ORA-28500: connection from ORACLE to a non-Oracle system returned this message:
[Generic Connectivity Using ODBC]DRV_GetRows: errors.h (1094): ;
[unixODBC][IBM][CLI Driver] CLI0165E Error in row.
SQLSTATE=01S01[unixODBC][IBM][CLI Driver] CLI0002W Data truncated.
SQLSTATE=01004[unixODBC][IBM][CLI Driver] CLI0002W Data truncated.
SQLSTATE=01004 (SQL State: 01004; SQL Code: -99999)
ORA-02063: preceding 2 lines from HSDB2UDB

The Oracle database is using a UNICODE character set, the column content matches the column precison and the table contains at least one character, for example German letters with diaeresis like ÜÖÄ.
For example the column is defined as a char(2) and it contains for example the value 'ÖB'

Solution 2:
The DB2 ODBC driver is per default using the character set of the application for the connection; see ODBC trace file:
SQLDriverConnect( hDbc=0:1, hwnd=0:0, szConnStrIn="DSN=toolsdb;UID=db2;PWD=***",
cbConnStrIn=-3, szConnStrOut=&bfffd0e0, cbConnStrOutMax=1020, pcbConnStrOut=&bf
ffd0de, fDriverCompletion=SQL_DRIVER_NOPROMPT )
---> Time elapsed - +5.600000E-005 seconds
( DBMS NAME="DB2/NT", Version="08.02.0000", Fixpack="0x23010106" )
( Application Codepage=1208, Database Codepage=1252, Char Send/Recv Codepage=12
08, Graphic Send/Recv Codepage=1200 )

Codepage 1208 is a unicode codepage.
HSODBC uses the BYTE NLS_LENGTH_SEMANTICS to transfer values from the foreign database to Oracle; a char(2) is then represented by char (2 bytes).
In a unicode codepage the 'Ö' is represented by 2 bytes, 'B' in one byte = 3 bytes.

To avoid this behaviour the environment variable DB2CODEPAGE can be set to any 8 bit character set which will then intialize the ODBC connection using the character set specified by DB2CODEPAGE.

As HSODBC does not know how to interprete the characters fetched from the DB2 database, in addition to the DB2CODEPAGE the HS_LANGUAGE must be set in the init<hsodbc>.ora file:

...
set DB2INSTANCE=db2inst1
set DB2CODEPAGE=819
HS_LANGUAGE=american_america.we8iso8859P1

HS_LANGUAGE must be set to a character set which matches the DB2CODEPAGE character set. An overview of those matching characters can be found at:

http://www.borgendale.com/codepage/j2conv.htm

References

NOTE:234517.1 - How to Resolve Common Errors Encountered while using Transparent Gateways or Generic Connectivity
NOTE:374744.1 - Using a Gateway with a Unicode Oracle Database Increases the Column Precision Three Times for Certain Data Types

Related

Products

Oracle Database Products > Oracle Database > Oracle Database > Oracle Server - Enterprise Edition

Errors

ORA-2063; ORA-28500

How To Use The Discoverer 'collectlogs' Utility [ID 259815.1]

2010-01-14T17:04:00.002+07:00



		Modified 16-OCT-2008 Type TROUBLESHOOTING Status PUBLISHED

In this Document
Purpose
Last Review Date
Instructions for the Reader
Troubleshooting Details
References

Applies to:

Oracle Discoverer - Version: 4.1.48.06 to 10.1.2.3
MS Windows 2000
Solaris Operating System (SPARC)
Microsoft Windows 2000Solaris Operating System (SPARC 32-bit)

Purpose

This document is intended for Discoverer Administrators and proxies (DBA, Application Server Administrators) to explain how to run the Collect Logs Utility for Discoverer and provide additional details to the documentation.

http://download-west.oracle.com/docs/html/B13918_03/tshoot.htm#i1028801
D.2.9 How to copy Discoverer log files

* The tool is present in $ORACLE_HOME/discoverer/util/ and is called collectlogs.sh in Unix/Linux or collectLogs.bat in the case of MS Windows.
* The tool will collect the middle tier Discoverer diagnostics and environment information and can useful for Oracle Support investigations.

Last Review Date

May 29, 2004

Instructions for the Reader

A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.

Troubleshooting Details

How to run CollectLogs on Discoverer 10g (10.1.2):

1. Connect to the Discoverer middle-tier.
2. Navigate to: $OH/discoverer
3. For Linux/Unix, set your environment by running: . discwb.sh
Not relevant for Windows.
4. Navigate to $OH/discoverer/util
5. For Linux/Unix run: collectlogs.sh <disco_logs.tar>
The <disco_logs.tar> parameter will be the output name of the file produced by the script.
6. For Windows run: collectlogs.bat

Additional information about how the script operates:

* Linux/Unix: A temporary folder called "discologs" is created in the util folder, all the files in this folder and its subfolders are merged into discologs.tar and the folder is removed.
* Windows a folder called collectlogs is created in the util folder and it contains all the files collected.
* Collectlogs will prompt asking for the EUL owner connect string and system TNS connect string. This is required to connect the database and collect database and EUL specific information.
* The tool collects:
o Discoverer log and configuration files
o Application Server log and configuration files
o Operating System server and environment information (CPU, memory, environment variables etc.)
o database and EUL information (db version, parameters, EUL objects export etc.).
* When collectlogs finishes, there is a file called <tarfile>.tar or on Windows a folder called collectlogs in the $ORACLE_HOME/discoverer/util directory.

It could have a larg size,so it is recommended zip the file before uploading to support.
For example, on Unix run: zip <zipfile>.zip <tarfile>.tar
* The file 'loglist.txt' file is an index of all the collected files listed by folder.
For example:

<Day Name> <Month> <day number> <timestamp> 
    
collectlogs tool has collected the following files in     
<$OH>/discoverer/util/discologs.tar     
    
-rw-r----- 1 ias904 dba 159 Sep 5 17:42 loglist.txt     
    
Apache:     
total 124     
-rwxr-x--x 1 ias904 dba 348 Sep 5 17:36 access.conf     
-rw-r----- 1 ias904 dba 491 Sep 5 17:36 aqxml.conf     
-rw-r----- 1 ias904 dba 1749 Sep 5 17:36 dms.conf     
-rw-r----- 1 ias904 dba 40704 Sep 5 17:36 httpd.conf     
-rw-r----- 1 ias904 dba 2534 Sep 5 17:36 mod_oc4j.conf     
-rw-r----- 1 ias904 dba 906 Sep 5 17:36 mod_osso.conf     
-rw-r----- 1 ias904 dba 1078 Sep 5 17:36 oracle_apache.conf     
-rw-r----- 1 ias904 dba 407 Sep 5 17:36 setupinfo.txt     
-rwxr-x--x 1 ias904 dba 357 Sep 5 17:36 srm.conf     
-rw-r----- 1 ias904 dba 8047 Sep 5 17:36 ssl.conf     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 ssl_engine_log     
-rw-r----- 1 ias904 dba 259 Sep 5 17:36 wireless_modrewrite.conf     
-rw-r----- 1 ias904 dba 940 Sep 5 17:36 wireless_ohs.conf     
    
checkdiscoverer:     
total 2     
-rw-r----- 1 ias904 dba 434 Sep 5 17:36 diag.log     
    
config:     
total 134     
-rw-r----- 1 ias904 dba 466 Sep 5 17:36 application.xml     
-rw-r----- 1 ias904 dba 14540 Sep 5 17:36 blaf.css     
-rw-r----- 1 ias904 dba 6047 Sep 5 17:36 blaf.htm     
-rw-r----- 1 ias904 dba 3217 Sep 5 17:36 configuration.xml     
-rw-r----- 1 ias904 dba 2967 Sep 5 17:36 discoverer.xsl     
-rwxr-x--- 1 ias904 dba 1376 Sep 5 17:36 discwb.sh     
-rw-r----- 1 ias904 dba 2719 Sep 5 17:36 graphproperties.xss     
-rw-r----- 1 ias904 dba 428 Sep 5 17:36 impl_rep     
-rw-r----- 1 ias904 dba 612 Sep 5 17:36 oadutil.txt     
-rw-r----- 1 ias904 dba 1938 Sep 5 17:36 orion-application.xml     
-rw-r----- 1 ias904 dba 1515 Sep 5 17:36 osfind.txt     
-rw-r----- 1 ias904 dba 8710 Sep 5 17:36 plus_config.xml     
-rw-r----- 1 ias904 dba 13119 Sep 5 17:36 ui_config.xml     
-rw-r----- 1 ias904 dba 2179 Sep 5 17:36 Versions.txt     
-rw-r----- 1 ias904 dba 1524 Sep 5 17:36 web.xml     
    
db:     
total 1310     
-rw-r----- 1 ias904 dba 69766 Sep 5 17:42 dbinfo.txt     
-rw-r----- 1 ias904 dba 574778 Sep 5 17:37 eul.eex     
-rw-r----- 1 ias904 dba 2459 Sep 5 17:37 eul.log     
-rw-r----- 1 ias904 dba 6844 Sep 5 17:38 eulinfo.txt     
    
dcm:     
total 4     
-rw-r----- 1 ias904 dba 180 Sep 5 17:36 dcm.conf     
drwxr-x--- 6 ias904 dba 512 Sep 5 17:36 logs     
    
dcm/logs:     
total 8     
drwxr-x--- 2 ias904 dba 512 Sep 5 17:36 busrule_logs     
drwxr-x--- 2 ias904 dba 512 Sep 5 17:36 daemon_logs     
drwxr-x--x 2 ias904 dba 512 Sep 5 17:36 dcmctl_logs     
drwxr-x--x 2 ias904 dba 512 Sep 5 17:36 emd_logs     
    
dcm/logs/busrule_logs:     
total 0     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 clustermanagement_busrule.log     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 deployment_busrule.log     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 localEdit_busrule.log     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 smi_busrule.log     
    
dcm/logs/daemon_logs:     
total 20     
-rw-r----- 1 ias904 dba 9666 Sep 5 17:36 log.xml     
    
dcm/logs/dcmctl_logs:     
total 6     
-rw-r----- 1 ias904 dba 2139 Sep 5 17:36 log.xml     
    
dcm/logs/emd_logs:     
total 0     
    
ldap:     
total 10     
-rw-r----- 1 ias904 dba 66 Sep 5 17:36 das.properties     
-rw-r----- 1 ias904 dba 159 Sep 5 17:36 groupattrs.properties     
-rw-r----- 1 ias904 dba 33 Sep 5 17:36 groupobjclasses.properties     
-rw-r----- 1 ias904 dba 181 Sep 5 17:36 oiddas.conf     
-rw-r----- 1 ias904 dba 178 Sep 5 17:36 oiddas.properties     
    
midtier:     
total 178     
-rw-r----- 1 ias904 dba 2967 Sep 5 17:36 env.txt     
-r--r----- 1 ias904 dba 136 Sep 5 17:36 hosts     
-rw-r----- 1 ias904 dba 1121 Sep 5 17:36 ias.properties     
-rw-r----- 1 ias904 dba 373 Sep 5 17:36 infratool.properties     
-rw-r----- 1 ias904 dba 16255 Sep 5 17:36 install.log     
-rw-r----- 1 ias904 dba 2551 Sep 5 17:36 midtier.txt     
-rw-r----- 1 ias904 dba 1203 Sep 5 17:36 portlist.ini     
-rw-r----- 1 ias904 dba 48639 Sep 5 17:36 showrev.txt     
-rw-r----- 1 ias904 dba 265 Sep 5 17:36 sqlnet.ora     
-rw-r----- 1 ias904 dba 12271 Sep 5 17:36 tnsnames.ora     
    
OC4J_BI_Forms:     
total 302     
-rw-r----- 1 ias904 dba 125411 Sep 5 17:36 application.log     
-rw-r----- 1 ias904 dba 3217 Sep 5 17:36 configuration.xml     
-rw-r----- 1 ias904 dba 969 Sep 5 17:36 data-sources.xml     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 default-web-access.log     
-rw-r----- 1 ias904 dba 2113 Sep 5 17:36 global-application.log     
-rw-r----- 1 ias904 dba 508 Sep 5 17:36 jms.log     
-rw-r----- 1 ias904 dba 403 Sep 5 17:36 oc4j.properties     
-rw-r----- 1 ias904 dba 1016 Sep 5 17:36 rmi.log     
-rw-r----- 1 ias904 dba 1637 Sep 5 17:36 server.log     
-rw-r----- 1 ias904 dba 1524 Sep 5 17:36 web.xml     
    
OC4J_Portal:     
total 28     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 default-web-access.log     
-rw-r----- 1 ias904 dba 9980 Sep 5 17:36 global-application.log     
-rw-r----- 1 ias904 dba 508 Sep 5 17:36 jms.log     
-rw-r----- 1 ias904 dba 1016 Sep 5 17:36 rmi.log     
-rw-r----- 1 ias904 dba 1631 Sep 5 17:36 server.log     
    
opmn:     
total 220     
-rw-r----- 1 ias904 dba 25942 Sep 5 17:36 Discoverer~OAD~1     
-rw-r----- 1 ias904 dba 153 Sep 5 17:36 Discoverer~OSAgent~1     
-rw-r----- 1 ias904 dba 597 Sep 5 17:36 Discoverer~PreferenceServer~1     
-rw-r----- 1 ias904 dba 20589 Sep 5 17:36 ipm.log     
-rw-r----- 1 ias904 dba 6810 Sep 5 17:36 OC4J~OC4J_BI_Forms~default_island~1     
-rw-r----- 1 ias904 dba 23645 Sep 5 17:36 OC4J~OC4J_Portal~default_island~1     
-rw-r----- 1 ias904 dba 14731 Sep 5 17:36 ons.log     
-rw-r----- 1 ias904 dba 378 Sep 5 17:36 opmn.log     
-rw-r----- 1 ias904 dba 13370 Sep 5 17:36 opmn.xml     
    
server:     
total 0     
    
util:     
total 144     
-rwxr-x--- 1 ias904 dba 375 Sep 5 17:36 applypreferences.sh     
-rwxr-x--- 1 ias904 dba 749 Sep 5 17:36 checkdiscoverer.sh     
-rwxr-x--- 1 ias904 dba 7797 Sep 5 17:36 collectlogs.sh     
-rw-r----- 1 ias904 dba 366 Sep 5 17:36 core.txt     
-rwxr-x--- 1 ias904 dba 12088 Sep 5 17:36 defaults.txt     
-rw-r----- 1 ias904 dba 434 Sep 5 17:36 diag.log     
-rw-r----- 1 ias904 dba 0 Sep 5 17:36 error.txt     
-rw-r----- 1 ias904 dba 571 Sep 5 17:36 exportDiscovererPortlet.sh     
-rw-r----- 1 ias904 dba 898 Sep 5 17:36 importDiscovererPortlet.sh     
-rw-r----- 1 ias904 dba 9877 Sep 5 17:36 loglist.txt     
-rwxr-x--- 1 ias904 dba 863 Sep 5 17:36 logparam.sh     
-rwxr-x--- 1 ias904 dba 570 Sep 5 17:36 migrateprefs.sh     
-rw-r----- 1 ias904 dba 468 Sep 5 17:36 oadj.ior     
-rw-r----- 1 ias904 dba 488 Sep 5 17:36 oadj_ior.txt     
-rwxr-x--- 1 ias904 dba 454 Sep 5 17:36 oadlist.sh     
-rwxr-x--- 1 ias904 dba 516 Sep 5 17:36 osfind.sh     
-rwxr-x--- 1 ias904 dba 12088 Sep 5 17:36 pref.txt     
-rwxr-x--- 1 ias904 dba 788 Sep 5 17:36 registerall.sh     
-rwxr-x--- 1 ias904 dba 1368 Sep 5 17:36 registersession.sh     
-rwxr-x--- 1 ias904 dba 550 Sep 5 17:36 restartoad.sh     
-rwxr-x--- 1 ias904 dba 429 Sep 5 17:36 runses.sh     
-rwxr-x--- 1 ias904 dba 210 Sep 5 17:36 ServerLogParams.txt     
-rw-r----- 1 ias904 dba 1272 Sep 5 17:36 sqlnet.log     
-rwxr-x--- 1 ias904 dba 366 Sep 5 17:36 startall.sh     
-rwxr-x--- 1 ias904 dba 1716 Sep 5 17:36 startoad.sh     
-rwxr-x--- 1 ias904 dba 990 Sep 5 17:36 startosagent.sh     
-rwxr-x--- 1 ias904 dba 1010 Sep 5 17:36 startpreference.sh     
-rwxr-x--- 1 ias904 dba 683 Sep 5 17:36 stopall.sh     
-rwxr-x--- 1 ias904 dba 921 Sep 5 17:36 stopoad.sh     
-rwxr-x--- 1 ias904 dba 810 Sep 5 17:36 stoposagent.sh     
-rwxr-x--- 1 ias904 dba 772 Sep 5 17:36 stoppreference.sh     
-rwxr-x--- 1 ias904 dba 747 Sep 5 17:36 unregistersession.sh     
    
webcache:     
total 3000     
-rw-r----- 1 ias904 dba 1493512 Sep 5 17:36 access_log     
-rw-r----- 1 ias904 dba 12030 Sep 5 17:36 event_log     
-rw------- 1 ias904 dba 1665 Sep 5 17:36 internal.xml     
-rw------- 1 ias904 dba 11454 Sep 5 17:36 webcache.xml     
-rw-r----- 1 ias904 dba 500 Sep 5 17:36 webcache_opmn.xml     
-rw-r----- 1 ias904 dba 1015 Sep 5 17:36 webcachetargets.xml     
    
xsl:     
total 528     
-rw-r----- 1 ias904 dba 12227 Sep 5 17:36 export.xsl     
-rw-r----- 1 ias904 dba 30947 Sep 5 17:36 lov_param_multiple_select.xsl     
-rw-r----- 1 ias904 dba 15119 Sep 5 17:36 lov_param_single_select.xsl     
-rw-r----- 1 ias904 dba 20023 Sep 5 17:36 parameters.xsl     
-rw-r----- 1 ias904 dba 40051 Sep 5 17:36 preferences.xsl     
-rw-r----- 1 ias904 dba 34172 Sep 5 17:36 presentation_options.xsl     
-rw-r----- 1 ias904 dba 5427 Sep 5 17:36 responsibility_list.xsl     
-rw-r----- 1 ias904 dba 14219 Sep 5 17:36 run_query_confirmation.xsl     
-rw-r----- 1 ias904 dba 10724 Sep 5 17:36 sorting.xsl     
-rw-r----- 1 ias904 dba 16186 Sep 5 17:36 workbook_list.xsl     
-rw-r----- 1 ias904 dba 51423 Sep 5 17:36 worksheet_display.xsl     
-rw-r----- 1 ias904 dba 13858 Sep 5 17:36 worksheet_layout.xsl

How To Run CollectLogs on Desupported Discoverer versions:

Note: Since the following versions are desupported, the following information is provided for information only. The information may or may not be updated or reviewed by Oracle Support for relevancy or accuracy.

* Discoverer 10g r1 (9.0.4): Run the same as 10g (10.1.2)
* Discoverer 9i (9.0.2): The script did not ship with the product. You may run the script below on Linux/Unix the same as 10g for your own review since the product is desupported.
* Discoverer 4i (4.1.48.08): This tool is shipped with 4.1.48 patch set and later, but works with any Discoverer 4.1 version. These are run in the same manner as the 10g scripts above; however, the script names are different:
1. Connect to the Discoverer middle-tier.
2. Navigate to: $OH/discwb4
3. For Linux/Unix, set your environment by running: . discwb.sh
Not relevant for Windows.
4. Navigate to $OH/discwb4/util
5. For Linux/Unix run: collectlogs4i.sh <disco_logs.tar>
The <disco_logs.tar> parameter will be the output name of the file produced by the script.
6. For Windows a working collectlogs script does not exist.

[script begins below line] - Discoverer 9i (9.0.2)    
===============================================================================     
#!/bin/ksh     
# Copyright (c) 2002 by Oracle Corporation. All Rights Reserved     
#     
#     
#     
# NAME     
# collectlogs.sh     
# FUNCTION     
# collect system info, iAS and Discoverer configutation and log files     
# NOTES     
#     
    
    
CURR_DIR=$PWD     
    
. $ORACLE_HOME/discoverer902/discwb.sh     
OH=$ORACLE_HOME     
    
    
if [ $# -le 0 ]     
then     
echo "Usage:collectlogs.sh <tarname>"     
echo " specify the tar file name to put the log files into"     
exit -1     
fi     
    
COLLECT_DIR_NAME=$1     
mkdir -p $COLLECT_DIR_NAME     
cd $COLLECT_DIR_NAME     
    
    
echo "Collecting FND Log Files"     
if test "$FND_DEBUG_DIR" = ""     
then     
echo "Unable to find FND_DEBUG_DIR "     
else     
mkdir fnd     
cd fnd     
cp $FND_DEBUG_DIR/* .     
cd ..     
fi     
echo "Done."     
    
echo "Collecting util folder files"     
mkdir util     
cd util     
cp $OH/discoverer902/util/*.sh .     
cp $OH/discoverer902/util/*.log .     
cp $OH/discoverer902/util/*.txt .     
cp $OH/discoverer902/util/*.ior .     
$OH/vbroker4/bin/printIOR oadj.ior > oadj_ior.txt 2>&1     
echo "Core file in the $ORACLE_HOME/discoverer902/util " "\n" > core.txt 2>&1     
echo "\n" "$ ls -l $OH/discoverer902/util/core" >> core.txt 2>&1     
ls -l $OH/discoverer902/util/core >> core.txt 2>&1     
echo "\n" "$ file $OH/discoverer902/util/core" >> core.txt 2>&1     
file $OH/discoverer902/util/core >> core.txt 2>&1     
cd ..     
echo "Done."     
    
    
echo "Collecting other discoverer config files"     
mkdir config     
cd config     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/common/xsl/blaf.css .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/common/xsl/blaf.htm .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/WEB-INF/configuration.xml .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/common/xsl/discoverer.xsl .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/plus_files/xsl/plus_config.xml .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/common/xsl/ui_config.xml .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/WEB-INF/web.xml .     
cp $OH/j2ee/OC4J_BI_Forms/application-deployments/discoverer/orion-application.xml .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/META-INF/application.xml .     
cp $OH/discoverer902/discwb.sh .     
cp $OH/discoverer902/bin/.reg/.reg* .     
cp $OH/vbroker4/adm/impl_dir/impl_rep .     
echo "Discoverer Versions - Binaries and Libraries" "\n" > Versions.txt 2>&1     
strings $OH/discoverer902/bin/dis51?? |grep "Discoverer Version" >> Versions.txt     
strings $OH/discoverer902/lib/lib*.so |grep "Discoverer Version" >> Versions.txt     
echo "\n" "$ ls -l $OH/discoverer902/bin/dis51* " >> Versions.txt 2>&1     
ls -l $OH/discoverer902/bin/dis51* >> Versions.txt 2>&1     
echo "\n" "$ ls -l $OH/discoverer902/lib " >> Versions.txt 2>&1     
ls -l $OH/discoverer902/lib >> Versions.txt 2>&1     
echo "Running Command oadutil list -VBJprop vbroker.agent.port=$OSAGENT_PORT"     
oadutil list -VBJprop vbroker.agent.port=$OSAGENT_PORT > oadutil.txt 2>&1     
echo "Running Command osfind -VBJprop vbroker.agent.port=$OSAGENT_PORT"     
osfind -VBJprop vbroker.agent.port=$OSAGENT_PORT > osfind.txt 2>&1     
cd ..     
echo "Done."     
    
echo "Collecting xsl discoverer config files"     
mkdir xsl     
cd xsl     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/viewer_files/xsl/* .     
cd ..     
echo "Done."     
    
echo "Collecting Middle Tier environment info"     
mkdir midtier     
cd midtier     
env > env.txt 2>&1     
cp $OH/install/portlist.ini .     
cp $OH/assistants/opca/install.log .     
cp $OH/config/ias.properties .     
cp $OH/config/infratool.properties .     
echo "Middle Tier Server" "\n" > midtier.txt 2>&1     
echo "$ uname -a" >> midtier.txt 2>&1     
uname -a >> midtier.txt 2>&1     
echo "\n" "$ /usr/sbin/psrinfo -v" >> midtier.txt 2>&1     
/usr/sbin/psrinfo -v >> midtier.txt 2>&1     
echo "\n" "$ /usr/sbin/prtconf | grep Memory" >> midtier.txt 2>&1     
/usr/sbin/prtconf | grep Memory >> midtier.txt 2>&1     
echo "\n" "$ df -k $OH" >> midtier.txt 2>&1     
df -k $OH >> midtier.txt 2>&1     
echo "\n" "$ ulimit -a" >> midtier.txt 2>&1     
ulimit -a >> midtier.txt 2>&1     
echo "\n" "$ umask" >> midtier.txt 2>&1     
umask >> midtier.txt 2>&1     
echo "\n" >> midtier.txt 2>&1     
echo "PATH = "$PATH >> midtier.txt 2>&1     
echo "\n" >> midtier.txt 2>&1     
echo "LD_LIBRARY_PATH = " $LD_LIBRARY_PATH >> midtier.txt 2>&1     
echo "\n" "$ /usr/sbin/ifconfig -a" >> midtier.txt 2>&1     
/usr/sbin/ifconfig -a >> midtier.txt 2>&1     
    
showrev -p >> showrev.txt 2>&1     
    
if test "$TNS_ADMIN" = ""     
then     
cp $OH/network/admin/sqlnet.ora .     
cp $OH/network/admin/tnsnames.ora .     
else     
cp $TNS_ADMIN/sqlnet.ora .     
cp $TNS_ADMIN/tnsnames.ora .     
fi     
    
echo "Copying /etc/hosts "     
cp /etc/hosts .     
    
cd ..     
echo "Done."     
    
echo "Collecting Apache config and log files"     
mkdir Apache     
cd Apache     
cp $OH/Apache/Apache/logs/*_log .     
cp $OH/Apache/Apache/conf/*.conf .     
cp $OH/Apache/Apache/setupinfo.txt .     
cd ..     
echo "Done."     
    
echo "Collecting OC4J_BI_Forms config and log files"     
mkdir OC4J_BI_Forms     
cd OC4J_BI_Forms     
cp $OH/j2ee/OC4J_BI_Forms/log/OC4J_BI_Forms_default_island_1/* .     
cp $OH/j2ee/OC4J_BI_Forms/application-deployments/discoverer/OC4J_BI_Forms_default_island_1/* .     
cp $OH/j2ee/OC4J_BI_Forms/config/data-sources.xml .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/WEB-INF/configuration.xml .     
cp $OH/j2ee/OC4J_BI_Forms/applications/discoverer/web/WEB-INF/web.xml .     
cp $OH/j2ee/OC4J_BI_Forms/config/oc4j.properties .     
cd ..     
echo "Done."     
    
echo "Collecting OC4J_Portal log files"     
mkdir OC4J_Portal     
cd OC4J_Portal     
cp $OH/j2ee/OC4J_Portal/log/OC4J_Portal_default_island_1/*.log .     
cd ..     
echo "Done."     
    
echo "Collecting OPMN config and log files"     
mkdir opmn     
cd opmn     
cp $OH/opmn/conf/opmn.xml .     
cp $OH/opmn/logs/*.log .     
cd ..     
echo "Done."     
    
echo "Collecting WebCache config and log files"     
mkdir webcache     
cd webcache     
cp $OH/webcache/*.xml .     
cp $OH/webcache/logs/*_log .     
cd ..     
echo "Done."     
    
echo "Collecting DCM config file"     
mkdir dcm     
cd dcm     
cp $OH/dcm/config/dcm.conf .     
cp -rf $OH/dcm/logs/ .     
cd ..     
echo "Done."     
    
echo "Collecting LDAP config file"     
mkdir ldap     
cd ldap     
cp $OH/ldap/das/*.properties .     
cp $OH/ldap/das/*.conf .     
cd ..     
echo "Done."     
    
clear     
echo "Collecting Database and EUL info"     
echo "Insert EUL owner connect string: "     
echo "Username: \c"     
read eulowner     
echo "Password: \c"     
stty -echo     
read password     
stty echo     
echo "\nDatabase: \c"     
read database     
echo "\n"     
mkdir db     
cd db     
echo "Getting EUL and db info..."     
$OH/bin/sqlplus $eulowner/$password@$database @$OH/discoverer902/util/dbinfo.sql     
cd ..     
echo "Done."     
    
    
    
echo "Listing all the collected logs into loglist.txt"     
date > loglist.txt 2>&1     
echo "\n" "collectlogs tool has collected the following files in " >> loglist.txt 2>&1     
echo $CURR_DIR/$COLLECT_DIR_NAME.tar "\n" >> loglist.txt 2>&1     
ls -lR * >> loglist.txt 2>&1     
cp loglist.txt ..     
echo "Done."     
    
    
cd ..     
    
tar cvf $COLLECT_DIR_NAME.tar $COLLECT_DIR_NAME     
echo "Removing the temp directory $COLLECT_DIR_NAME"     
rm -rf $COLLECT_DIR_NAME     
echo "ALL the informations are available in "     
echo $CURR_DIR/$COLLECT_DIR_NAME.tar     
echo "loglist.txt lists the collected files"     
echo "Done."     
    
===============================================================================     
[script ends above line]

References

NOTE:228524.1 - How To Use Discoverer 'checkdiscoverer' Utility in 4.1 release

How To Enable Tracing For Discoverer 10g (10.1) Portlet Issues [ID 370337.1]

2010-01-14T17:00:00.002+07:00



		Modified 09-OCT-2008 Type TROUBLESHOOTING Status PUBLISHED

In this Document
Goal
Solution

Applies to:

Oracle Discoverer - Version: 10.1.2 to 10.1.2
Information in this document applies to any platform.

Goal

Explain in simple steps, how to create a diagnostic trace for Discoverer 10g (10.1.2) PortletProvider to diagnost portlet issues or provide to Oracle Support.

Solution

Stop the middle-tier (opmnctl stopall) or OC4J_BI_Forms and Discoverer at a minimum.
Delete (or rename and delete if you prefer to keep) the applicaion.log file:
$ORACLE_HOME/j2ee/OC4J_BI_Forms/application-deployments/discoverer/OC4J_BI_Forms_default_island_1/application.log

Note: you will have one applicaton.log for each JVM you have configured.
For example:OC4J_BI_Forms_default_island_1, OC4J_BI_Forms_default_island_2

Use the following command to find all the Discoverer application.log(s).

cd $ORACLE_HOME
find . -name 'application.log' | grep discoverer
Set the PortletProvider logLevel to "trace" and diagnostic framework logkey to "disco.portlet.debug".
Edit: $ORACLE_HOME/discoverer/config/configuration.xml

change line: FROM:  <portlet logKey="disco.portlet.fatal" logLevel="error" pickerCompatibleVersions="10.1.2" maxDataRows="1000"> TO:  <portlet logKey="disco.portlet.debug" logLevel="trace" pickerCompatibleVersions="10.1.2" maxDataRows="1000">
Start the middle-tier (opmnctl startall) or stopped components.
Immediately reproduce the problem.
Review or upload the application.log

How To Verify Your Discoverer 10g Session (dis51ws) and Preference (dis51pr) Executables Respond Properly [ID 561165.1]

2010-01-14T16:57:00.002+07:00



		Modified 19-MAR-2009 Type TROUBLESHOOTING Status PUBLISHED

In this Document
Purpose
Last Review Date
Instructions for the Reader
Troubleshooting Details

Applies to:

Oracle Discoverer - Version: 10.1.2.0.0 to 10.1.2.3
Information in this document applies to any platform.

Purpose

This document describes how to check your Discoverer 10g (10.1.2) middle-tier session (di51ws) and preference (dis51pr) executables to ensure they are responding properly.

This is a helpful check if you get errors when starting the Discoverer components via the Oracle Process Manager Notification (OPMN) service or you receive errors when accessing Discoverer Plus or Viewer (possibly after a patch application or some other system change).

Last Review Date

April 7, 2008

Instructions for the Reader

A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.

Troubleshooting Details

Unix / Linux:

Set your environment middle-tier environment ($ORACLE_HOME)

export ORACLE_HOME=<middle-tier path> To verify: echo $ORACLE_HOME
Type: cd $ORACLE_HOME/discoverer
Set the Discoverer environment by running:

$. ./discwb.sh (notice: dot<space>dot<slash>scriptname
Type: cd $ORACLE_HOME/bin
Check the permissions with the following command:

$ ls -ltr dis51* -rwxr-x--- 1 oracle oinstall 44372603 Feb 13 17:49 dis51ws -rwxr-x--- 1 oracle oinstall 1540376 Feb 13 17:49 dis51pr

Ensure that dis51ws and dis51pr both have proper execute permissions. If they do not, then grant them with:

chmod 750 dis51ws chmod 750 dis51pr
Check that the Discoverer session server responds with the following command:

Type: ./dis51ws

You should see an output similar to the following:

$ ./dis51ws Registering SessionManager Setting up the Operation Mode and Application Name Setting the creator of DCSThreadMonitor Starting the memory manager

The executable will appear to hang at this point. This is normal since we are not passing it any further parameters.
To exit the test, type: Control + C on the keyboard.
Check that the Discoverer preference server responds with the following command:

Type: ./dis51pr

You should see an output similar to the following:

$ ./dis51pr Creating Preference Repository: Activating Preference Repository Preference Repository is ready... IOR:000000000000001a49444c3a444349434f524241496e746572666163653a312e30000000000000010000000000000090000102000000001b6f726c6473732d61697830312e75732e6f7261636c652e636f6d000094ee00000000003114010f004e555000000014000000000000000100446973636f7665726572000000000000000001507265666572656e63650000000000000200000000000000080000000054414f0000000001000000140000000000010001000000000001010900000000

The executable will appear to hang at this point. This is normal since we are not passing it any further parameters.
To exit the test, type: Control + C on the keyboard.
You have verified the executables are responding properly.

Windows:

Open a DOS prompt.
Set the ORACLE_HOME

set oracle_home=<middle-tier path> To verify: echo %oracle_home%
Type: cd %oracle_home%\bin
Check that the Discoverer session server responds with the following command:

prompt>dis51ws.exe

You should see an output similar to the following:

C:\bihome\bin>dis51ws Registering SessionManager Setting up the Operation Mode and Application Name Setting the creator of DCSThreadMonitor Starting the memory manager
To exit the test, type: Control + C on the keyboard.
Check that the Discoverer preference server responds with the following command:

promt>dis51pr.exe

You should see an output similar to the following:

C:\bihome\bin>dis51pr Creating Preference Repository: Activating Preference Repository Preference Repository is ready... IOR:010000001a00000049444c3a444349434f524241496e746572666163653a312e3000000001000000000000008c000000010102001a0000006f72 6c6473732d323030332e75732e6f7261636c652e636f6d009a083100000014010f004e555000000014000000000100000000446973636f7665726572 000000000001000000507265666572656e636500000002000000000000000800000001000000004f4154010000001400000001000000010001000000 00000901010000000000
To exit the test, type: Control + C on the keyboard.
You have verified the executables are responding properly.

How to Relink Oracle Database Software on UNIX [ID 131321.1]

2010-01-13T16:31:00.002+07:00


PURPOSE
-------

Provide relinking instructions for Oracle Database software
on UNIX platforms.

 
SCOPE & APPLICATION
-------------------

Anyone who maintains Oracle RDBMS software on a UNIX platform.


Relinking Oracle
================

Background:
Applications for UNIX are generally not distributed as complete executables.  
Oracle, like many application vendors who create products for UNIX, distribute 
individual object files, library archives of object files, and some source 
files which then get "relinked" at the operating system level during 
installation to create usable executables.  This guarantees a reliable
integration with functions provided by the OS system libraries.

Relinking occurs automatically under these circumstances:

 - An Oracle product has been installed with an Oracle provided installer.
 - An Oracle patch set has been applied via an Oracle provided installer.  

The following information has been added to the 'Certify' section of Metalink:

   General Notes For Oracle Database - Enterprise Edition:
   O/S Information: 
   The vendors guarantee operating system binary compatibility; therefore, no 
   reinstall or relink of the Oracle software is required when upgrading these
   operating systems unless specifically stated otherwise.

Relinking Oracle manually is suggested under the following circumstances
(even though the OS vendor may not require it):

 - An OS upgrade has occurred.
 - A change has been made to the OS system libraries.  This can occur during 
   the application of an OS patch.
 - A new install failed during the relinking phase.
 - Individual Oracle executables core dump during initial startup.
 - An individual Oracle patch has been applied (however, explicit relink 
   instructions are usually either included in the README or integrated into 
   the patch install script) 

Customers who want to (or have been advised to) manually relink should follow 
the procedure below.

[Step 1] Log into the UNIX system as the Oracle software owner
==============================================================================
Typically this is the user 'oracle'.
 

[STEP 2] Verify that your $ORACLE_HOME is set correctly:
===============================================================================
For all Oracle Versions and Platforms, perform this basic environment check 
first:
 
 % cd $ORACLE_HOME
 % pwd    

 ...Doing this will ensure that $ORACLE_HOME is set correctly in your current 
    environment.
 

[Step 3] Verify and/or Configure the UNIX Environment for Proper Relinking:
===============================================================================
For all Oracle Versions and UNIX Platforms:
 The Platform specific environment variables LIBPATH, LD_LIBRARY_PATH, & 
 SHLIB_PATH typically are already set to include system library locations like
 '/usr/lib'.  In most cases, you need only check what they are set to first, 
 then add the $ORACLE_HOME/lib directory to them where appropriate.
 i.e.:  % setenv LD_LIBRARY_PATH ${ORACLE_HOME}/lib:${LD_LIBRARY_PATH}
 (see Note:131207.1 How to Set UNIX Environment Variables for help with 
 setting UNIX environment variables)

If on AIX with:
--------------
    
    NOTE: Before relinking, after stopping all required processes, issue the
    following as root to detach all files from memory that are not attached 
    to an active process:

         # /usr/sbin/slibclean

    Oracle 7.3.X:
       - Set LIBPATH to include $ORACLE_HOME/lib
    
    Oracle 8.0.X:
       - Set LIBPATH to include $ORACLE_HOME/lib
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib and 
         $ORACLE_HOME/network/lib (Required when using Oracle products that 
         use Java)
       - Set LINK_CNTRL to L_PTHREADS_D7 if using AIX 4.3. ('oslevel' verifies 
         OS version)
    
    Oracle 8.1.X, 9.X.X, 10.X.X or 11.x.x:
       - For 8.1.5, set LINK_CNTRL to L_PTHREADS_D7
       - If not 8.1.5, ensure that LINK_CNTRL is not set
       - Set LIBPATH to include $ORACLE_HOME/lib
      

If on DATA GENERAL AVIION (DG) with:
-----------------------------------

    Oracle 7.3.X or 8.0.X:
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib
       - ensure TARGET_BINARY_INTERFACE is unset
    
    Oracle 8.1.X:
       - Set LD_LIBRARY_PATH to include 
         $ORACLE_HOME/lib:$ORACLE_HOME/JRE/lib/PentiumPro/native_threads

If on HP-UX with:
----------------

    Oracle 7.3.X, 8.0.X, 8.1.X;
       - Set SHLIB_PATH to $ORACLE_HOME/lib
       If using 64bit 8i Oracle, also
       - Set LD_LIBRARY_PATH to $ORACLE_HOME/lib64
       - ensure LPATH is unset
 
    Oracle 9.X.X, 10.X.X or 11.x.x;
       - ensure LPATH is unset

If on NCR with:
--------------

    Oracle 7.3.X, 8.0.X or 8.1.X:
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib:/usr/ccs/lib

If on SCO UNIXware with:
-----------------------

    Oracle 7.3.X or 8.0.X:
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib
    
    Oracle 8.1.X:
       - Set LD_LIBRARY_PATH to include 
         $ORACLE_HOME/lib:$ORACLE_HOME/JRE/lib/x86at/native_threads

If on SGI with:
--------------

    32bit Oracle 7.3.X or 8.0.X:
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib
       - Set SGI_ABI to -32  

    64bit Oracle 8.0.X or 8.1.X (8i is only available in 64bit):
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib
       - Set SGI_ABI to -64
       - If one does not already exist, create the file compiler.defaults and 
         set the COMPILER_DEFAULTS_PATH variable:

     In the Oracle software owner's $HOME directory, create a file called
     'compiler.defaults':
  
        % cd $HOME
        % echo "-DEFAULT:abi=64:isa=mips3:proc=r10k" > compiler.defaults

     Then set the environment variable COMPILER_DEFAULTS_PATH to point to the
     $HOME directory.
  
        % setenv COMPILER_DEFAULTS_PATH $HOME
  
     If this is not set, relinking will fail because the compiler defaults to
     MIPS4 objects although Oracle requires MIPS3.
       - Set LD_LIBRARY64_PATH to include the $ORACLE_HOME/lib and the 
         $ORACLE_HOME/javavm/admin directories. 
       - Set LD_LIBRARYN32_PATH to include the $ORACLE_HOME/lib32 directory. 
         NOTE: LD_LIBRARY64_PATH & LD_LIBRARYN32_PATH must be undefined when 
         installing software with Oracle Universal Installer. 

If on SOLARIS (Sparc or Intel) with:
------------------------------------

    Oracle 7.3.X, 8.0.X, or 8.1.X:
       - Ensure that /usr/ccs/bin is before /usr/ucb in $PATH  
         % which ld   ....should return '/usr/ccs/bin/ld'

       If using 32bit(pre 9i) Oracle, 
       - Set LD_LIBRARY_PATH=$ORACLE_HOME/lib

       If using 64bit(pre 9i) Oracle, 
       - Set LD_LIBRARY_PATH=$ORACLE_HOME/lib
       - Set LD_LIBRARY_PATH_64=$ORACLE_HOME/lib64

    Oracle 9.X.X or higher:
       - LD_LIBRARY_PATH & LD_LIBRARY_PATH_64 do not need to be set to include
         a reference to $ORACLE_HOME/lib or $ORACLE_HOME/lib64 for a generic database
         software installation. 
         (However they should not contain a reference to the ORACLE_HOME of another Oracle version)


If on Digital/Tru64, IBM/Sequent PTX, Linux or any other UNIX Platform not 
mentioned above with:
------------------------------------------------------------------------------

    Oracle 7.3.X, 8.0.X, 8.1.X, 9.X.X, 10.X.X or 11.x.x:
       - Set LD_LIBRARY_PATH to include $ORACLE_HOME/lib


[Step 4] For all Oracle Versions and UNIX Platforms: 
===============================================================================
Verify that you performed Step 2 correctly:

 % env|pg  ....make sure that you see the correct absolute path for 
   $ORACLE_HOME in the variable definitions.
 

[Step 5] For all Oracle Versions and UNIX Platforms: 
===============================================================================
Verify umask is set correctly:

 % umask

This must return 022.  If it does not, set umask to 022.

 % umask 022
 % umask

[Step 6] Run the OS Commands to Relink Oracle: 
===============================================================================
Important Note:  Before relinking Oracle, shut down both the database and the 
                 listener.
Important Note:  The following commands will output a lot of text to your 
                 session window.  To capture this output for upload to support,
                 redirect the output to a file.
Important Note:  If relinking a client installation, it's expected that some 
                 aspects of the following commands will fail if the components
                 were not originally installed.

For all UNIX platforms:

Oracle 7.3.x 
------------
 For executables:  oracle, exp, imp, sqlldr, tkprof 

      % cd $ORACLE_HOME/rdbms/lib 
      % make -f ins_rdbms.mk install

 For executables:  svrmgrl, svrmgrm 

      % cd $ORACLE_HOME/svrmgr/lib 
      % make -f ins_svrmgr.mk linstall minstall      <- linstall is for svrmgrl, 
                                                        minstall is for svrmgrm

 For executables:  sqlplus 

      % cd $ORACLE_HOME/sqlplus/lib 
      % make -f ins_sqlplus.mk install

 For executables:  dbsnmp, oemevent, oratclsh 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_agent.mk install

 For executables:  names, namesctl 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_names.mk install

 For executables:  tnslsnr, lsnrctl, tnsping, csmnl, trceval, trcroute 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_network.mk install


Oracle 8.0.x 
------------
 For executables:  oracle, exp, imp, sqlldr, tkprof, mig, dbv, orapwd, rman, 
                   svrmgrl, ogms, ogmsctl 

      % cd $ORACLE_HOME/rdbms/lib 
      % make -f ins_rdbms.mk install

 For executables:  sqlplus 

      % cd $ORACLE_HOME/sqlplus/lib 
      % make -f ins_sqlplus.mk install

 For executables:  dbsnmp, oemevent, oratclsh, libosm.so 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_oemagent.mk install

 For executables:  tnslsnr, lsnrctl, namesctl, names, osslogin, trcasst, 
                   trcroute 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_network.mk install


Oracle 8.1.X, 9.X.X, 10.X.X or 11.X.X
-------------------------------------
  *** NEW IN 8i AND ABOVE ***

   A 'relink' script is provided in the $ORACLE_HOME/bin directory.
     % cd $ORACLE_HOME/bin
     % relink      ...this will display all of the command's options.
       usage: relink <parameter>
       accepted values for parameter: all, oracle, network, client, 
       client_sharedlib, interMedia, precomp, utilities, oemagent, ldap 
  
  Note: ldap option is available only from 9i. In 8i, you would have to manually relink
  ldap.

  You can relink most of the executables associated with an Oracle Server Installation
  by running the following command:
     % relink all    
  This will not relink every single executable Oracle provides(you can
  discern which executables were relinked by checking their timestamp with
  'ls -l' in the $ORACLE_HOME/bin directory).  However, 'relink all' will 
  recreate the shared libraries that most executables rely on and thereby
  resolve most issues that require a proper relink. 

 
11.2.X special instructions
--------------------------------------
   In 11.2 relink script will only accept "all" as argument


 -or-

  Since the 'relink' command merely calls the traditional 'make' commands, you 
  still have the option of running the 'make' commands independently:

 For executables:  oracle, exp, imp, sqlldr, tkprof, mig, dbv, orapwd, rman, 
                   svrmgrl, ogms, ogmsctl 

      % cd $ORACLE_HOME/rdbms/lib 
      % make -f ins_rdbms.mk install

      NOTE: After relinking the oracle executable, make sure that the 
      permissions on the executable are 6751 (-rwsr-s--x). If they are
      not, run the following command as the Oracle software owner:

      % cd $ORACLE_HOME/bin
      % chmod 6751 oracle

 For executables:  sqlplus 

      % cd $ORACLE_HOME/sqlplus/lib 
      % make -f ins_sqlplus.mk install

 For executables:  isqlplus   (Oracle9i and higher versions)

      % cd $ORACLE_HOME/sqlplus/lib 
      % make -f ins_sqlplus.mk install_isqlplus

 For executables:  dbsnmp, oemevent, oratclsh 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_oemagent.mk install

      NOTE: After relinking the dbsnmp executable, it is necessary to run
      the following commands as root (so that the ownership/permissions on
      the executable are correct):

      # cd $ORACLE_HOME/bin
      # chown root dbsnmp
      # chmod 6750 dbsnmp

      @ References: Note:233559.1 and Bug:2858326

 For executables:  names, namesctl 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_names.mk install

 For executables:  osslogin, trcasst, trcroute, onrsd, tnsping 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_net_client.mk install

 For executables:  tnslsnr, lsnrctl 

      % cd $ORACLE_HOME/network/lib 
      % make -f ins_net_server.mk install

 For executables related to ldap (for example Oracle Internet Directory):

      % cd $ORACLE_HOME/ldap/lib
      % make -f ins_ldap.mk install

How to Tell if Relinking Was Successful:
===============================================================================
If relinking was successful, the make command will eventually return to the OS 
prompt without an error. There will NOT be a 'Relinking Successful' type
message.


If You Receive an Error Message During Relinking:
===============================================================================
Confirm that the message you received is an actual fatal error and not a 
warning. Relinking errors usually terminate the relinking process and contain 
verbage similar to the following:
'Fatal error', 'Ld: fatal', 'Exit Code 1'
While warnings will look similar to: 'ld: warning: option -YP appears more than
once, first setting taken' and can most often be ignored.

If you receive an error that terminates the relinking process, your first step 
should be to extract the relevant information about the error from the make 
output:

This can be broken down into three basic steps:
 1. Identify the OS utility that is returning the error.
    'ld', 'make', 'cc', 'mv', 'cp', 'ar' are common sources.
 2. Identify the type of error:
    'Permission Denied', 'Undefined Symbol', 'File Not Found' are common types.
 3. Identify the files or symbols involved.

Using the information from above as keywords, search Oracle's Metalink 
repository ( http://metaLink.oracle.com ) for previous occurrences of the same 
error. If no previous occurances are found or a solution is not provided, 
generate an iTAR that includes the complete error text.

Help setting environment variables.
==============================================================================
See Note:131207.1 How to Set UNIX Environment Variables 
for help with setting UNIX environment variables.


Relinking with Orainst:
===============================================================================
For Oracle7 & Oracle8 only, the following document illustrates how to relink 
with the 'orainst' utility:
   Note:1032747.6 HOW TO RELINK ORACLE USING THE 7.3.X INSTALLER
While 'orainst' will run the same commands as [Step 4], performing [Step 4] 
manually from a UNIX shell is the preferred approach.


RELATED DOCUMENTS
-----------------

Note:131207.1   How to Set UNIX Environment Variables
Note:109621.1   HP/UX: LD_LIBRARY_PATH and SHLIB_PATH
Note:1032747.6  HOW TO RELINK ORACLE USING THE 7.3.X INSTALLER
Bug:1337908     THE $ORACLE_HOME/BIN/RELINK SCRIPT DOES NOT RELINK EXP, IMP 
                  SQLLOADER

Client Requests Error Out or Time Out (IIS 6.0)

2010-01-06T11:04:00.002+07:00

In IIS 6.0, settings are set to aggressive and secure defaults to minimize attacks due to timeouts and limits that were previously too generous. IIS enforces the following timeouts at the connection level:

•Limits on response buffering. The metabase property ASPBufferingLimit is 4 MB, by default. If ASP scripts buffer more than this, they generate an error. Prior to IIS 6.0, there was no limit to buffering.

•Limits on posts. The metabase property AspMaxRequestEntityAllowed enforces a maximum ASP POST size of 204800 bytes, with each individual field limited to 100 KB. Prior to IIS 6.0, there was no limit to posts.

•The ServerListenTimeoutproperty is no longer available. The metabase property ServerListenTimeout has been replaced by the following metabase properties:

•ConnectionTimeout. This property specifies the amount of time, in seconds, that the server waits before disconnecting an inactive connection.

•MinFileBytesPerSec. When IIS responds to a client request, the MinFileBytesPerSec property determines the length of time that the client has to receive the entire response. If the client computer takes too long to receive the entire response, the kernel-mode driver, HTTP.sys, terminates the connection according to the timeout value.

•HeaderWaitTimeout. When a client connects to the Web server, the client computer is given a time limit to send in all headers for the request (demarked by a final double \r\n). If the complete header set for the request is not received within the time period indicated by HeaderWaitTimeout, HTTP.sys resets the connection. You can configure the value of HeaderWaitTimeout.

•Header size limitation. By default, HTTP.sys accepts only requests where the size of the request header is less than 16 KB. This means that if HTTP.sys does not receive the terminating carriage return/line feed (CRLF) pair sequence within 16 KB, HTTP.sys considers the request malicious and terminates the connection. You can change the header size limitation by adjusting the value of the registry entry MaxRequestBytes. Note that this entry does not exist in the registry by default. To add it to the registry, use the registry editor, Regedit.exe.

Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning

2009-11-27T13:22:00.006+07:00

1. Introduction

The Java™ Platform, Standard Edition (Java SE™) is used for a wide variety of applications, from small applets on desktops to web services on large servers. In support of this diverse range of deployments, the Java HotSpot™ virtual machine implementation (Java HotSpot™ VM) provides multiple garbage collectors, each designed to satisfy different requirements. This is an important part of meeting the demands of both large and small applications. However, users, developers and administrators that need high performance are burdened with the extra step of selecting the garbage collector that best meets their needs. A significant step toward removing this burden was made in J2SE™ 5.0: the garbage collector is selected based on the class of the machine on which the application is run.

This better choice of the garbage collector is generally an improvement, but is by no means always the best choice for every application. Users with strict performance goals or other requirements may need to explicitly select the garbage collector and tune certain parameters to achieve the desired level of performance. This document provides information to help with those tasks. First, the general features of a garbage collector and basic tuning options are described in the context of the serial, stop-the-world collector. Then specific features of the other collectors are presented along with factors to consider when selecting a collector.

When does the choice of a garbage collector matter? For some applications, the answer is never. That is, the application can perform well in the presence of garbage collection with pauses of modest frequency and duration. However, this is not the case for a large class of applications, particularly those with large amounts of data (multiple gigabytes), many threads and high transaction rates.

Amdahl observed that most workloads cannot be perfectly parallelized; some portion is always sequential and does not benefit from parallelism. This is also true for the Java™ platform. In particular, virtual machines from Sun Microsystems for the Java platform prior to J2SE 1.4 do not support parallel garbage collection, so the impact of garbage collection on a multiprocessor system grows relative to an otherwise parallel application.

The graph below models an ideal system that is perfectly scalable with the exception of garbage collection. The red line is an application spending only 1% of the time in garbage collection on a uniprocessor system. This translates to more than a 20% loss in throughput on 32 processor systems. At 10% of the time in garbage collection (not considered an outrageous amount of time in garbage collection in uniprocessor applications) more than 75% of throughput is lost when scaling up to 32 processors.

This shows that negligible speed issues when developing on small systems may become principal bottlenecks when scaling up to large systems. However, small improvements in reducing such a bottleneck can produce large gains in performance. For a sufficiently large system it becomes well worthwhile to select the right garbage collector and to tune it if necessary.

The serial collector is usually adequate for most "small" applications (those requiring heaps of up to approximately 100MB on modern processors). The other collectors have additional overhead and/or complexity which is the price for specialized behavior. If the application doesn't need the specialized behavior of an alternate collector, use the serial collector. An example of a situation where the serial collector is not expected to be the best choice is a large application that is heavily threaded and run on a machine with a large amount of memory and two or more processors. When applications are run on such server-class machines, the parallel collector is selected by default (see Ergonomics below).

This document was developed using Java SE 6 on the Solaris™ Operating System (SPARC^(R) Platform Edition) as the reference. However, the concepts and recommendations presented here apply to all supported platforms, including Linux, Microsoft Windows and the Solaris Operating System (x86 Platform Edition). In addition, the command line options mentioned are available on all supported platforms, although the default values of some options may be different on each platform.

2. Ergonomics

A feature referred to here as ergonomics was introduced in J2SE 5.0. The goal of ergonomics is to provide good performance with little or no tuning of command line options by selecting the

garbage collector,
heap size,
and runtime compiler

at JVM startup, instead of using fixed defaults. This selection assumes that the class of the machine on which the application is run is a hint as to the characteristics of the application (i.e., large applications run on large machines). In addition to these selections is a simplified way of tuning garbage collection. With the parallel collector the user can specify goals for a maximum pause time and a desired throughput for an application. This is in contrast to specifying the size of the heap that is needed for good performance. This is intended to particularly improve the performance of large applications that use large heaps. The more general ergonomics is described in the document entitled “Ergonomics in the 5.0 Java Virtual Machine”. It is recommended that the ergonomics as presented in this latter document be tried before using the more detailed controls explained in this document.

Included in this document are the ergonomics features provided as part of the adaptive size policy for the parallel collector. This includes the options to specify goals for the performance of garbage collection and additional options to fine tune that performance.

3. Generations

One strength of the J2SE platform is that it shields the developer from the complexity of memory allocation and garbage collection. However, once garbage collection is the principal bottleneck, it is worth understanding some aspects of this hidden implementation. Garbage collectors make assumptions about the way applications use objects, and these are reflected in tunable parameters that can be adjusted for improved performance without sacrificing the power of the abstraction.

An object is considered garbage when it can no longer be reached from any pointer in the running program. The most straightforward garbage collection algorithms simply iterate over every reachable object. Any objects left over are then considered garbage. The time this approach takes is proportional to the number of live objects, which is prohibitive for large applications maintaining lots of live data.

Beginning with the J2SE 1.2, the virtual machine incorporated a number of different garbage collection algorithms that are combined using generational collection. While naive garbage collection examines every live object in the heap, generational collection exploits several empirically observed properties of most applications to minimize the work required to reclaim unused ("garbage") objects. The most important of these observed properties is the weak generational hypothesis, which states that most objects survive for only a short period of time.

The blue area in the diagram below is a typical distribution for the lifetimes of objects. The X axis is object lifetimes measured in bytes allocated. The byte count on the Y axis is the total bytes in objects with the corresponding lifetime. The sharp peak at the left represents objects that can be reclaimed (i.e., have "died") shortly after being allocated. Iterator objects, for example, are often alive for the duration of a single loop.

Some objects do live longer, and so the distribution stretches out to the the right. For instance, there are typically some objects allocated at initialization that live until the process exits. Between these two extremes are objects that live for the duration of some intermediate computation, seen here as the lump to the right of the initial peak. Some applications have very different looking distributions, but a surprisingly large number possess this general shape. Efficient collection is made possible by focusing on the fact that a majority of objects "die young."

To optimize for this scenario, memory is managed in generations, or memory pools holding objects of different ages. Garbage collection occurs in each generation when the generation fills up. The vast majority of objects are allocated in a pool dedicated to young objects (the young generation), and most objects die there. When the young generation fills up it causes a minor collection in which only the young generation is collected; garbage in other generations is not reclaimed. Minor collections can be optimized assuming the weak generational hypothesis holds and most objects in the young generation are garbage and can be reclaimed. The costs of such collections are, to the first order, proportional to the number of live objects being collected; a young generation full of dead objects is collected very quickly. Typically some fraction of the surviving objects from the young generation are moved to the tenured generation during each minor collection. Eventually, the tenured generation will fill up and must be collected, resulting in a major collection, in which the entire heap is collected. Major collections usually last much longer than minor collections because a significantly larger number of objects are involved.

As noted above, ergonomics selects the garbage collector dynamically in order to provide good performance on a variety of applications. The serial garbage collector is designed for applications with small data sets and its default parameters were chosen to be effective for most small applications. The throughput garbage collector is meant to be used with applications that have medium to large data sets. The heap size parameters selected by ergonomics plus the features of the adaptive size policy are meant to provide good performance for server applications. These choices work well in most, but not all, cases. Which leads to the central tenet of this document:

If garbage collection becomes a bottleneck, you will most likely have to customize the total heap size as well as the sizes of the individual generations. Check the verbose garbage collector output and then explore the sensitivity of your individual performance metric to the garbage collector parameters.

The default arrangement of generations (for all collectors with the exception of the parallel collector) looks something like this.

At initialization, a maximum address space is virtually reserved but not allocated to physical memory unless it is needed. The complete address space reserved for object memory can be divided into the young and tenured generations.

The young generation consists of eden and two survivor spaces. Most objects are initially allocated in eden. One survivor space is empty at any time, and serves as the destination of any live objects in eden and the other survivor space during the next copying collection. Objects are copied between survivor spaces in this way until they are old enough to be tenured (copied to the tenured generation).

A third generation closely related to the tenured generation is the permanent generation which holds data needed by the virtual machine to describe objects that do not have an equivalence at the Java language level. For example objects describing classes and methods are stored in the permanent generation.

Performance Considerations

There are two primary measures of garbage collection performance:

Throughput is the percentage of total time not spent in garbage collection, considered over long periods of time. Throughput includes time spent in allocation (but tuning for speed of allocation is generally not needed).
Pauses are the times when an application appears unresponsive because garbage collection is occurring.

Users have different requirements of garbage collection. For example, some consider the right metric for a web server to be throughput, since pauses during garbage collection may be tolerable, or simply obscured by network latencies. However, in an interactive graphics program even short pauses may negatively affect the user experience.

Some users are sensitive to other considerations. Footprint is the working set of a process, measured in pages and cache lines. On systems with limited physical memory or many processes, footprint may dictate scalability. Promptness is the time between when an object becomes dead and when the memory becomes available, an important consideration for distributed systems, including remote method invocation (RMI).

In general, a particular generation sizing chooses a trade-off between these considerations. For example, a very large young generation may maximize throughput, but does so at the expense of footprint, promptness and pause times. young generation pauses can be minimized by using a small young generation at the expense of throughput. To a first approximation, the sizing of one generation does not affect the collection frequency and pause times for another generation.

There is no one right way to size generations. The best choice is determined by the way the application uses memory as well as user requirements. Thus the virtual machine's choice of a garbage collectior is not always optimal and may be overridden with command line options described below.

Measurement

Throughput and footprint are best measured using metrics particular to the application. For example, throughput of a web server may be tested using a client load generator, while footprint of the server might be measured on the Solaris Operating System using the pmap command. On the other hand, pauses due to garbage collection are easily estimated by inspecting the diagnostic output of the virtual machine itself.

The command line option -verbose:gc causes information about the heap and garbage collection to be printed at each collection. For example, here is output from a large server application:


[GC 325407K->83000K(776768K), 0.2300771 secs]
[GC 325816K->83372K(776768K), 0.2454258 secs]
[Full GC 267628K->83769K(776768K), 1.8479984 secs]

Here we see two minor collections followed by one major collection. The numbers before and after the arrow (e.g., 325407K->83000K from the first line) indicate the combined size of live objects before and after garbage collection, respectively. After minor collections the size includes some objects that are garbage (no longer alive) but that cannot be reclaimed. These objects are either contained in the tenured generation, or referenced from the tenured or permanent generations.

The next number in parentheses (e.g., (776768K) again from the first line) is the committed size of the heap: the amount of space usable for java objects without requesting more memory from the operating system. Note that this number does not include one of the survivor spaces, since only one can be used at any given time, and also does not include the permanent generation, which holds metadata used by the virtual machine.

The last item on the line (e.g., 0.2300771 secs) indicates the time taken to perform the collection; in this case approximately a quarter of a second.

The format for the major collection in the third line is similar.

The format of the output produced by -verbose:gc is subject to change in future releases.

The option -XX:+PrintGCDetails causes additional information about the collections to be printed. An example of the output with -XX:+PrintGCDetails using the serial garbage collector is shown here.


[GC [DefNew: 64575K->959K(64576K), 0.0457646 secs] 196016K->133633K(261184K), 0.0459067 secs]

indicates that the minor collection recovered about 98% of the young generation, DefNew: 64575K->959K(64576K) and took 0.0457646 secs (about 45 milliseconds).

The usage of the entire heap was reduced to about 51% 196016K->133633K(261184K) and that there was some slight additional overhead for the collection (over and above the collection of the young generation) as indicated by the final time of 0.0459067 secs.

The option -XX:+PrintGCTimeStamps will add a time stamp at the start of each collection. This is useful to see how frequently garbage collections occur.

111.042: [GC 111.042: [DefNew: 8128K->8128K(8128K), 0.0000505 secs]111.042: [Tenured: 18154K->2311K(24576K), 0.1290354 secs] 26282K->2311K(32704K), 0.1293306 secs]

The collection starts about 111 seconds into the execution of the application. The minor collection starts at about the same time. Additionally the information is shown for a major collection delineated by Tenured. The tenured generation usage was reduced to about 10% 18154K->2311K(24576K) and took 0.1290354 secs (approximately 130 milliseconds).

As was the case with -verbose:gc, the format of the output produced by -XX:+PrintGCDetails is subject to change in future releases.

4. Sizing the Generations

A number of parameters affect generation size. The following diagram illustrates the difference between committed space and virtual space in the heap. At initialization of the virtual machine, the entire space for the heap is reserved. The size of the space reserved can be specified with the -Xmx option. If the value of the -Xms parameter is smaller than the value of the -Xmx parameter, not all of the space that is reserved is immediately committed to the virtual machine. The uncommitted space is labeled "virtual" in this figure. The different parts of the heap (permanent generation, tenured generation and young generation) can grow to the limit of the virtual space as needed.

Some of the parameters are ratios of one part of the heap to another. For example the parameter NewRatio denotes the relative size of the tenured generation to the young generation. These parameters are discussed below.

Total Heap

Note that the following discussion regarding growing and shrinking of the heap and default heap sizes does not apply to the parallel collector. (See the section on ergonomics for details on heap resizing and default heap sizes with the parallel collector.) However, the parameters that control the total size of the heap and the sizes of the generations do apply to the parallel collector.

Since collections occur when generations fill up, throughput is inversely proportional to the amount of memory available. Total available memory is the most important factor affecting garbage collection performance.

By default, the virtual machine grows or shrinks the heap at each collection to try to keep the proportion of free space to live objects at each collection within a specific range. This target range is set as a percentage by the parameters -XX:MinHeapFreeRatio=<minimum> and -XX:MaxHeapFreeRatio=<maximum>, and the total size is bounded below by -Xms<min> and above by -Xmx<max>. The default parameters for the 32-bit Solaris Operating System (SPARC Platform Edition) are shown in this table:

Parameter	Default Value
`MinHeapFreeRatio`	`40`
`MaxHeapFreeRatio`	`70`
`-Xms`	`3670k`
`-Xmx`	`64m`

Default values of heap size parameters on 64-bit systems have been scaled up by approximately 30%. This increase is meant to compensate for the larger size of objects on a 64-bit system.

With these parameters, if the percent of free space in a generation falls below 40%, the generation will be expanded to maintain 40% free space, up to the maximum allowed size of the generation. Similarly, if the free space exceeds 70%, the generation will be contracted so that only 70% of the space is free, subject to the minimum size of the generation.

Large server applications often experience two problems with these defaults. One is slow startup, because the initial heap is small and must be resized over many major collections. A more pressing problem is that the default maximum heap size is unreasonably small for most server applications. The rules of thumb for server applications are:

Unless you have problems with pauses, try granting as much memory as possible to the virtual machine. The default size (64MB) is often too small.

Setting -Xms and -Xmx to the same value increases predictability by removing the most important sizing decision from the virtual machine. However, the virtual machine is then unable to compensate if you make a poor choice.

In general, increase the memory as you increase the number of processors, since allocation can be parallelized.

For reference, there is a separate page explaining some of the available command-line options.

The Young Generation

The second most influential knob is the proportion of the heap dedicated to the young generation. The bigger the young generation, the less often minor collections occur. However, for a bounded heap size a larger young generation implies a smaller tenured generation, which will increase the frequency of major collections. The optimal choice depends on the lifetime distribution of the objects allocated by the application.

By default, the young generation size is controlled by NewRatio. For example, setting -XX:NewRatio=3 means that the ratio between the young and tenured generation is 1:3. In other words, the combined size of the eden and survivor spaces will be one fourth of the total heap size.

The parameters NewSize and MaxNewSize bound the young generation size from below and above. Setting these to the same value fixes the young generation, just as setting -Xms and -Xmx to the same value fixes the total heap size. This is useful for tuning the young generation at a finer granularity than the integral multiples allowed by NewRatio.

Survivor Space Sizing

If desired, the parameter SurvivorRatio can be used to tune the size of the survivor spaces, but this is often not as important for performance. For example, -XX:SurvivorRatio=6 sets the ratio between eden and a survivor space to 1:6. In other words, each survivor space will be one sixth the size of eden, and thus one eighth the size of the young generation (not one seventh, because there are two survivor spaces).

If survivor spaces are too small, copying collection overflows directly into the tenured generation. If survivor spaces are too large, they will be uselessly empty. At each garbage collection the virtual machine chooses a threshold number of times an object can be copied before it is tenured. This threshold is chosen to keep the survivors half full. The command-line option -XX:+PrintTenuringDistribution can be used to show this threshold and the ages of objects in the new generation. It is also useful for observing the lifetime distribution of an application.

Here are the default values for the 32-bit Solaris Operating System (SPARC Platform Edition); the default values on other platforms are different.

	Default Value
Parameter	Client JVM	Server JVM
`NewRatio`	`8`	`2`
`NewSize`	`2228K`	`2228K`
`MaxNewSize`	`not limited`	`not limited`
`SurvivorRatio`	`32`	`32`

The maximum size of the young generation will be calculated from the maximum size of the total heap and NewRatio. The "not limited" default value for MaxNewSize means that the calculated value is not limited by MaxNewSize unless a value for MaxNewSize is specified on the command line.

The rules of thumb for server applications are:

First decide the maximum heap size you can afford to give the virtual machine. Then plot your performance metric against young generation sizes to find the best setting.
- Note that the maximum heap size should always be smaller than the amount of memory installed on the machine, to avoid excessive page faults and thrashing.

If the total heap size is fixed, increasing the young generation size requires reducing the tenured generation size. Keep the tenured generation large enough to hold all the live data used by the application at any given time, plus some amount of slack space (10-20% or more).

Subject to the above constraint on the tenured generation:
- Grant plenty of memory to the young generation.
- Increase the young generation size as you increase the number of processors, since allocation can be parallelized.

5. Available Collectors

The discussion to this point has been about the serial collector. The Java HotSpot VM includes three different collectors, each with different performance characteristics.

The serial collector uses a single thread to perform all garbage collection work, which makes it relatively efficient since there is no communication overhead between threads. It is best-suited to single processor machines, since it cannot take advantage of multiprocessor hardware, although it can be useful on multiprocessors for applications with small data sets (up to approximately 100MB). The serial collector is selected by default on certain hardware and operating system configurations, or can be explicitly enabled with the option -XX:+UseSerialGC.

The parallel collector (also known as the throughput collector) performs minor collections in parallel, which can significantly reduce garbage collection overhead. It is intended for applications with medium- to large-sized data sets that are run on multiprocessor or multi-threaded hardware. The parallel collector is selected by default on certain hardware and operating system configurations, or can be explicitly enabled with the option -XX:+UseParallelGC.
- New: parallel compaction is a feature introduced in J2SE 5.0 update 6 and enhanced in Java SE 6 that allows the parallel collector to perform major collections in parallel. Without parallel compaction, major collections are performed using a single thread, which can significantly limit scalability. Parallel compaction is enabled by adding the option -XX:+UseParallelOldGC to the command line.

The concurrent collector performs most of its work concurrently (i.e., while the application is still running) to keep garbage collection pauses short. It is designed for applications with medium- to large-sized data sets for which response time is more important than overall throughput, since the techniques used to minimize pauses can reduce application performance. The concurrent collector is enabled with the option -XX:+UseConcMarkSweepGC.

Selecting a Collector

Unless your application has rather strict pause time requirements, first run your application and allow the VM to select a collector. If necessary, adjust the heap size to improve performance. If the performance still does not meet your goals, then use the following guidelines as a starting point for selecting a collector.

If the application has a small data set (up to approximately 100MB), then
- select the serial collector with -XX:+UseSerialGC.

If the application will be run on a single processor and there are no pause time requirements, then
- let the VM select the collector, or
- select the serial collector with -XX:+UseSerialGC.

If (a) peak application performance is the first priority and (b) there are no pause time requirements or pauses of one second or longer are acceptable, then
- let the VM select the collector, or
- select the parallel collector with -XX:+UseParallelGC and (optionally) enable parallel compaction with -XX:+UseParallelOldGC.

If response time is more important than overall throughput and garbage collection pauses must be kept shorter than approximately one second, then
- select the concurrent collector with -XX:+UseConcMarkSweepGC. If only one or two processors are available, consider using incremental mode, described below.

These guidelines provide only a starting point for selecting a collector because performance is dependent on the size of the heap, the amount of live data maintained by the application and the number and speed of available processors. Pause times are particularly sensitive to these factors, so the threshold of one second mentioned above is only approximate: the parallel collector will experience pause times longer than one second on many data size and hardware combinations; conversely, the concurrent collector may not be able to keep pauses shorter than one second on some combinations.

If the recommended collector does not achieve the desired performance, first attempt to adjust the heap and generation sizes to meet the desired goals. If still unsuccessful, then try a different collector: use the concurrent collector to reduce pause times and use the parallel collector to increase overall throughput on multiprocessor hardware.

The Parallel Collector

The parallel collector (also referred to here as the throughput collector) is a generational collector similar to the serial collector; the primary difference is that multiple threads are used to speed up garbage collection. The parallel collector is enabled with the command line option -XX:+UseParallelGC. By default, only minor collections are executed in parallel; major collections are performed with a single thread. However, parallel compaction can be enabled with the option -XX:+UseParallelOldGC so that both minor and major collections are executed in parallel, to further reduce garbage collection overhead.

On a machine with N processors the parallel collector uses N garbage collector threads; however, this number can be adjusted with a command line option (see below). On a host with one processor, the parallel collector will likely not perform as well as the serial collector because of the overhead required for parallel execution (e.g., synchronization). However, when running applications with medium- to large-sized heaps, it generally outperforms the serial collector by a modest amount on machines with two processors, and usually performs significantly better than the serial collector when more than two processors are available.

The number of garbage collector threads can be controlled with the command line option -XX:ParallelGCThreads=<N>. If explicit tuning of the heap is being done with command line options, the size of the heap needed for good performance with the parallel collector is to first order the same as needed with the serial collector. Enabling the parallel collector should just make the minor collection pauses shorter. Because there are multiple garbage collector threads participating in the minor collection there is a small possibility of fragmentation due to promotions from the young generation to the tenured generation during the collection. Each garbage collection thread reserves a part of the tenured generation for promotions and the division of the available space into these "promotion buffers" can cause a fragmentation effect. Reducing the number of garbage collector threads will reduce this fragmentation effect as will increasing the size of the tenured generation.

Generations

As mentioned earlier, the arrangement of the generations is different in the parallel collector. That arrangement is shown in the figure below.

Ergonomics

Starting in J2SE 5.0, the parallel collector is selected by default on server-class machines as detailed in the document Garbage Collector Ergonomics. In addition, the parallel collector uses a method of automatic tuning that allows desired behaviors to be specified instead of generation sizes and other low-level tuning details. The behaviors that can be specified are:

Maximum garbage collection pause time

Throughput

Footprint (i.e., heap size)

The maximum pause time goal is specified with the command line option -XX:MaxGCPauseMillis=<N>. This is interpreted as a hint that pause times of <N> milliseconds or less are desired; by default there is no maximum pause time goal. If a pause time goal is specified, the heap size and other garbage collection related parameters are adjusted in an attempt to keep garbage collection pauses shorter than the specified value. Note that these adjustments may cause the garbage collector to reduce the overall throughput of the application and in some cases the desired pause time goal cannot be met.

The throughput goal is measured in terms of the time spent doing garbage collection vs. the time spent outside of garbage collection (referred to as application time). The goal is specified by the command line option -XX:GCTimeRatio=<N>, which sets the ratio of garbage collection time to application time to 1 / (1 + <N>).

For example, -XX:GCTimeRatio=19 sets a goal of 1/20 or 5% of the total time in garbage collection. The default value is 99, resulting in a goal of 1% of the time in garbage collection.

Maxmimum heap footprint is specified using the existing option -Xmx<N>. In addition, the collector has an implicit goal of minimizing the size of the heap as long as the other goals are being met.

Priority of goals

The goals are addressed in the following order

Maximum pause time goal

Throughput goal

Minimum footprint goal

The maximum pause time goal is met first. Only after it is met is the throughput goal addressed. Similarly, only after the first two goals have been met is the footprint goal considered.

Generation Size Adjustments

The statistics such as average pause time kept by the collector are updated at the end of each collection. The tests to determine if the goals have been met are then made and any needed adjustments to the size of a generation is made. The exception is that explicit garbage collections (e.g., calls to System.gc()) are ignored in terms of keeping statistics and making adjustments to the sizes of generations.

Growing and shrinking the size of a generation is done by increments that are a fixed percentage of the size of the generation so that a generation steps up or down toward its desired size. Growing and shrinking are done at different rates. By default a generation grows in increments of 20% and shrinks in increments of 5%. The percentage for growing is controlled by the command line flag -XX:YoungGenerationSizeIncrement=<Y> for the young generation and -XX:TenuredGenerationSizeIncrement=<T> for the tenured generation. The percentage by which a generation shrinks is adjusted by the command line flag -XX:AdaptiveSizeDecrementScaleFactor=<D>. If the growth increment is X percent, the decrement for shrinking is X / D percent.

If the collector decides to grow a generation at startup, there is a supplemental percentage added to the increment. This supplement decays with the number of collections and there is no long term affect of this supplement. The intent of the supplement is to increase startup performance. There is no supplement to the percentage for shrinking.

If the maximum pause time goal is not being met, the size of only one generation is shrunk at a time. If the pause times of both generations are above the goal, the size of the generation with the larger pause time is shrunk first.

If the throughput goal is not being met, the sizes of both generations are increased. Each is increased in proportion to its respective contribution to the total garbage collection time. For example, if the garbage collection time of the young generation is 25% of the total collection time and if a full increment of the young generation would be by 20%, then the young generation would be increased by 5%.

Default Heap Size

If not otherwise set on the command line, the initial and maximum heap sizes are calculated based on the amount of memory on the machine. The proportion of memory to use for the heap is controlled by the command line options DefaultInitialRAMFraction and DefaultMaxRAMFraction, as shown in the table below. (In the table, memory represents the amount of memory on the machine.)

	Formula	Default
initial heap size	`memory / DefaultInitialRAMFraction`	`memory / 64`
maximum heap size	`MIN(memory / DefaultMaxRAMFraction, 1GB)`	`MIN(memory / 4, 1GB)`

Note that the default maximum heap size will not exceed 1GB, regardless of how much memory is installed on the machine.

Excessive GC Time and OutOfMemoryError

The parallel collector will throw an OutOfMemoryError if too much time is being spent in garbage collection: if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered, an OutOfMemoryError will be thrown. This feature is designed to prevent applications from running for an extended period of time while making little or no progress because the heap is too small. If necessary, this feature can be disabled by adding the option -XX:-UseGCOverheadLimit to the command line.

Measurements

The verbose garbage collector output from the parallel collector is essentially the same as that from the serial collector.

7. The Concurrent Collector

The concurrent collector is designed for applications that prefer shorter garbage collection pauses and that can afford to share processor resources with the garbage collector while the application is running. Typically applications which have a relatively large set of long-lived data (a large tenured generation), and run on machines with two or more processors tend to benefit from the use of this collector. However, this collector should be considered for any application with a low pause time requirement; for example, good results have been observed for interactive applications with tenured generations of a modest size on a single processor, especially if using incremental mode. The concurrent collector is enabled with the command line option -XX:+UseConcMarkSweepGC.

Similar to the other available collectors, the concurrent collector is generational; thus both minor and major collections occur. The concurrent collector attempts to reduce pause times due to major collections by using separate garbage collector threads to trace the reachable objects concurrently with the execution of the application threads. During each major collection cycle, the concurrent collector will pause all the application threads for a brief period at the beginning of the collection and again toward the middle of the collection. The second pause tends to be the longer of the two pauses and multiple threads are used to do the collection work during that pause. The remainder of the collection including the bulk of the tracing of live objects and sweeping of unreachable objects is done with one or more garbage collector threads that run concurrently with the application. Minor collections can interleave with an on-going major cycle, and are done in a manner similar to the parallel collector (in particular, the application threads are stopped during minor collections).

The basic algorithms used by the concurrent collector are described in the technical report A Generational Mostly-concurrent Garbage Collector. Note that precise implementation details may, however, differ slightly as the collector is enhanced from one release to another.

Overhead of Concurrency

The concurrent collector trades processor resources (which would otherwise be available to the application) for shorter major collection pause times. The most visible overhead is the use of one or more processors during the concurrent parts of the collection. On an N processor system, the concurrent part of the collection will use K/N of the available processors, where 1 <= K <= ceiling{N/4}. (Note that the precise choice of and bounds on K are subject to change.) In addition to the use of processors during concurrent phases, additional overhead is incurred to enable concurrency. Thus while garbage collection pauses are typically much shorter with the concurrent collector, application throughput also tends to be slightly lower than with the other collectors.

On a machine with more than one processing core, there are processors available for application threads during the concurrent part of the collection, so the concurrent garbage collector thread does not "pause" the application. This usually results in shorter pauses, but again fewer processor resources are available to the application and some slowdown should be expected, especially if the application utilizes all of the processing cores maximally. Up to a limit, as N increases the reduction in processor resources due to concurrent garbage collection becomes smaller, and the benefit from concurrent collection increases. The following section, concurrent mode failure, discusses potential limits to such scaling.

Since at least one processor is utilized for garbage collection during the concurrent phases, the concurrent collector does not normally provide any benefit on a uniprocessor (single-core) machine. However, there is a separate mode available that can achieve low pauses on systems with only one or two processors; see incremental mode below for details.

Concurrent Mode Failure

The concurrent collector uses one or more garbage collector threads that run simultaneously with the application threads with the goal of completing the collection of the tenured and permanent generations before either becomes full. As described above, in normal operation, the concurrent collector does most of its tracing and sweeping work with the application threads still running, so only brief pauses are seen by the application threads. However, if the concurrent collector is unable to finish reclaiming the unreachable objects before the tenured generation fills up, or if an allocation cannot be satisfied with the available free space blocks in the tenured generation, then the application is paused and the collection is completed with all the application threads stopped. The inability to complete a collection concurrently is referred to as concurrent mode failure and indicates the need to adjust the concurrent collector parameters.

Excessive GC Time and OutOfMemoryError

The concurrent collector will throw an OutOfMemoryError if too much time is being spent in garbage collection: if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered, an OutOfMemoryError will be thrown. This feature is designed to prevent applications from running for an extended period of time while making little or no progress because the heap is too small. If necessary, this feature can be disabled by adding the option -XX:-UseGCOverheadLimit to the command line.

The policy is the same as that in the parallel collector, except that time spent performing concurrent collections is not counted toward the 98% time limit. In other words, only collections performed while the application is stopped count toward excessive GC time. Such collections are typically due to a concurrent mode failure or an explicit collection request (e.g., a call to System.gc()).

Floating Garbage

The concurrent collector, like all the other collectors in HotSpot, is a tracing collector that identifies at least all the reachable objects in the heap. In the parlance of Jones and Lins it is an incremental update collector. Because application threads and the garbage collector thread run concurrently during a major collection, objects that are traced by the garbage collector thread may subsequently become unreachable by the time collection finishes. Such unreachable objects that have not yet been reclaimed are referred to as floating garbage. The amount of floating garbage depends on the duration of the concurrent collection cycle and on the frequency of reference updates, also known as mutations, by the application. Furthermore, since the young generation and the tenured generation are collected independently, each acts a source of roots to the other. As a rough rule of thumb, try increasing the size of the tenured generation by 20% to account for the floating garbage. Floating garbage in the heap at the end of one concurrent collection cycle is collected during the next collection cycle.

Pauses

The concurrent collector pauses an application twice during a concurrent collection cycle. The first pause is to mark as live the objects directly reachable from the roots (e.g., object references from application thread stacks and registers, static objects and so on) and from elsewhere in the heap (e.g., the young generation). This first pause is referred to as the initial mark pause. The second pause comes at the end of the concurrent tracing phase and finds objects that were missed by the concurrent tracing due to updates by the application threads of references in an object after the concurrent collector had finished tracing that object. This second pause is referred to as the remark pause.

Concurrent Phases

The concurrent tracing of the reachable object graph occurs between the initial mark pause and the remark pause. During this concurrent tracing phase one or more concurrent garbage collector threads may be using processor resources that would otherwise have been available to the application and, as a result, compute-bound applications may see a commensurate fall in application throughput during this and other concurrent phases even though the application threads are not paused. After the remark pause, there is a concurrent sweeping phase which collects the objects identified as unreachable. Once a collection cycle completes, the concurrent collector will wait, consume almost no computational resources, until the start of the next major collection cycle.

Starting a Concurrent Collection Cycle

With the serial collector a major collection occurs whenever the tenured generation becomes full and all application threads are stopped while the collection is done. In contrast, a concurrent collection needs to be started at a time such that the collection can finish before the tenured generation becomes full; otherwise the application would observe longer pauses due to concurrent mode failure. There are several ways a concurrent collection can be started.

Based on recent history, the concurrent collector maintains estimates of the time remaining before the tenured generation will be exhausted and of the time needed for a concurrent collection cycle. Based on these dynamic estimates, a concurrent collection cycle will be started with the aim of completing the collection cycle before the tenured generation is exhausted. These estimates are padded for safety, since the concurrent mode failure can be very costly.

A concurrent collection will also start if the occupancy of the tenured generation exceeds an initiating occupancy, a percentage of the tenured generation. The default value of this initiating occupancy threshold is approximately 92%, but the value is subject to change from release to release. This value can be manually adjusted using the command line option

-XX:CMSInitiatingOccupancyFraction=<N>

where <N> is an integral percentage (0-100) of the tenured generation size.

Scheduling Pauses

The pauses for the young generation collection and the tenured generation collection occur independently. They do not overlap, but may occur in quick succession such that the pause from one collection, immediately followed by one from the other collection, can appear to be a single, longer pause. To avoid this, the concurrent collector attempts to schedule the remark pause roughly midway between the previous and next young generation pauses. This scheduling is currently not done for the initial mark pause, which is usually much shorter than the remark pause.

Incremental Mode

The concurrent collector can be used in a mode in which the concurrent phases are done incrementally. Recall that during a concurrent phase the garbage collector thread is using one or more processors. The incremental mode is meant to lessen the impact of long concurrent phases by periodically stopping the concurrent phase to yield back the processor to the application. This mode, referred to here as “i-cms,” divides the work done concurrently by the collector into small chunks of time which are scheduled between young generation collections. This feature is useful when applications that need the low pause times provided by the concurrent collector are run on machines with small numbers of processors (e.g., 1 or 2).

The concurrent collection cycle typically includes the following steps:

stop all application threads and identify the set of objects reachable from roots, then resume all application threads

concurrently trace the reachable object graph, using one or more processors, while the application threads are executing

concurrently retrace sections of the object graph that were modified since the tracing in the previous step, using one processor

stop all application threads and retrace sections of the roots and object graph that may have been modified since they were last examined, then resume all application threads

concurrently sweep up the unreachable objects to the free lists used for allocation, using one processor

concurrently resize the heap and prepare the support data structures for the next collection cycle, using one processor

Normally, the concurrent collector uses one or more processors during the entire concurrent tracing phase, without voluntarily relinquishing them. Similarly, one processor is used for the entire concurrent sweep phase, again without relinquishing it. This overhead can be too much of a disruption for applications with response time constraints that might otherwise have utilized the processing cores, particularly when run on systems with just one or two processors. Incremental mode solves this problem by breaking up the concurrent phases into short bursts of activity, which are scheduled to occur mid-way between minor pauses.

i-cms uses a duty cycle to control the amount of work the concurrent collector is allowed to do before voluntarily giving up the processor. The duty cycle is the percentage of time between young generation collections that the concurrent collector is allowed to run. i-cms can automatically compute the duty cycle based on the behavior of the application (the recommended method, known as automatic pacing), or the duty cycle can be set to a fixed value on the command line.

Command Line Options

The following command-line options control i-cms (see below for recommendations for an initial set of options):

Option	Description	Default Value
Option	Description	J2SE 5.0 and earlier	Java SE 6 and later
`-XX:+CMSIncrementalMode`	Enables incremental mode. Note that the concurrent collector must also be enabled (with -XX:+UseConcMarkSweepGC) for this option to work.	disabled	disabled
`-XX:+CMSIncrementalPacing`	Enables automatic pacing. The incremental mode duty cycle is automatically adjusted based on statistics collected while the JVM is running.	disabled	enabled
`-XX:CMSIncrementalDutyCycle=<N>`	The percentage (0-100) of time between minor collections that the concurrent collector is allowed to run. If CMSIncrementalPacing is enabled, then this is just the initial value.	50	10
`-XX:CMSIncrementalDutyCycleMin=<N>`	The percentage (0-100) which is the lower bound on the duty cycle when CMSIncrementalPacing is enabled.	10	0
`-XX:CMSIncrementalSafetyFactor=<N>`	The percentage (0-100) used to add conservatism when computing the duty cycle.	10	10
`-XX:CMSIncrementalOffset=<N>`	The percentage (0-100) by which the incremental mode duty cycle is shifted to the right within the period between minor collections.	0	0
`-XX:CMSExpAvgFactor=<N>`	The percentage (0-100) used to weight the current sample when computing exponential averages for the concurrent collection statistics.	25	25

Recommended Options

To use i-cms in Java SE 6, use the following command line options:

-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode \

-XX:+PrintGCDetails -XX:+PrintGCTimeStamps

The first two options enable the concurrent collector and i-cms, respectively. The last two options are not required; they simply cause diagnostic information about garbage collection to be written to stdout, so that garbage collection behavior can be seen and later analyzed.

Note that in J2SE 5.0 and earlier releases, we recommend the following as an initial set of command line options for i-cms:

-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode \

-XX:+PrintGCDetails -XX:+PrintGCTimeStamps \

-XX:+CMSIncrementalPacing -XX:CMSIncrementalDutyCycleMin=0

-XX:CMSIncrementalDutyCycle=10

These are the same as recommended for Java SE 6, with the addition of three options that control i-cms automatic pacing. The additional options simply specify the values that became the default in Java SE 6.

Basic Troubleshooting

The i-cms automatic pacing feature uses statistics gathered while the program is running to compute a duty cycle so that concurrent collections complete before the heap becomes full. However, past behavior is not a perfect predictor of future behavior and the estimates may not always be accurate enough to prevent the heap from becoming full. If too many full collections occur, try the following steps, one at a time:

Step	Options
1. Increase the safety factor:	`-XX:CMSIncrementalSafetyFactor=<N>`
2. Increase the minimum duty cycle:	`-XX:CMSIncrementalDutyCycleMin=<N>`
3. Disable automatic pacing and use a fixed duty cycle:	`-XX:-CMSIncrementalPacing -XX:CMSIncrementalDutyCycle=<N>`

Measurements

Below is the output from the concurrent collector with the options -verbose:gc -XX:+PrintGCDetails, with a few minor details removed. Note that the output for the concurrent collector is interspersed with the output from the minor collections; typically many minor collections occur during a concurrent collection cycle. The CMS-initial-mark: indicates the start of the concurrent collection cycle. The CMS-concurrent-mark: indicates the end of the concurrent marking phase and CMS-concurrent-sweep: marks the end of the concurrent sweeping phase. Not discussed before is the precleaning phase indicated by CMS-concurrent-preclean:. Precleaning represents work that can be done concurrently in preparation for the remark phase CMS-remark. The final phase is indicated by the CMS-concurrent-reset: and is in preparation for the next concurrent collection.


[GC [1 CMS-initial-mark: 13991K(20288K)] 14103K(22400K), 0.0023781 secs]
[GC [DefNew: 2112K->64K(2112K), 0.0837052 secs] 16103K->15476K(22400K), 0.0838519 secs]
...
[GC [DefNew: 2077K->63K(2112K), 0.0126205 secs] 17552K->15855K(22400K), 0.0127482 secs]
[CMS-concurrent-mark: 0.267/0.374 secs]
[GC [DefNew: 2111K->64K(2112K), 0.0190851 secs] 17903K->16154K(22400K), 0.0191903 secs]
[CMS-concurrent-preclean: 0.044/0.064 secs]
[GC [1 CMS-remark: 16090K(20288K)] 17242K(22400K), 0.0210460 secs]
[GC [DefNew: 2112K->63K(2112K), 0.0716116 secs] 18177K->17382K(22400K), 0.0718204 secs]
[GC [DefNew: 2111K->63K(2112K), 0.0830392 secs] 19363K->18757K(22400K), 0.0832943 secs]
...
[GC [DefNew: 2111K->0K(2112K), 0.0035190 secs] 17527K->15479K(22400K), 0.0036052 secs]
[CMS-concurrent-sweep: 0.291/0.662 secs]
[GC [DefNew: 2048K->0K(2112K), 0.0013347 secs] 17527K->15479K(27912K), 0.0014231 secs]
[CMS-concurrent-reset: 0.016/0.016 secs]
[GC [DefNew: 2048K->1K(2112K), 0.0013936 secs] 17527K->15479K(27912K), 0.0014814 secs]

The initial mark pause is typically short relative to the minor collection pause time. The concurrent phases (concurrent mark, concurrent preclean and concurrent sweep) normally last significantly longer than a minor collection pause, as indicated by the example output above. Note, however, that the application is not paused during these concurrent phases. The remark pause is often comparable in length to a minor collection. The remark pause is affected by certain application characteristics (e.g., a high rate of object modification can increase this pause) and the time since the last minor collection (i.e., more objects in the young generation may increase this pause).

8. Other Considerations

Permanent Generation Size

The permanent generation does not have a noticeable impact on garbage collector performance for most applications. However, some applications dynamically generate and load many classes; for example, some implementations of JavaServer Pages (JSP) pages. These applications may need a larger permanent generation to hold the additional classes. If so, the maximum permanent generation size can be increased with the command-line option -XX:MaxPermSize=<N>.

Finalization; Weak, Soft and Phantom References

Some applications interact with garbage collection by using finalization and weak, soft, or phantom references. These features can create performance artifacts at the Java programming language level. An example of this is relying on finalization to close file descriptors, which makes an external resource (descriptors) dependent on garbage collection promptness. Relying on garbage collection to manage resources other than memory is almost always a bad idea.

The Resources section includes an article that discusses in depth some of the pitfalls of finalization and techniques for avoiding them.

Explicit Garbage Collection

Another way applications can interact with garbage collection is by invoking full garbage collections explicitly by calling System.gc(). This can force a major collection to be done when it may not be necessary (i.e., when a minor collection would suffice), and so in general should be avoided. The performance impact of explicit garbage collections can be measured by disabling them using the flag -XX:+DisableExplicitGC, which causes the VM to ignore calls to System.gc().

One of the most commonly encountered uses of explicit garbage collection occurs with RMI's distributed garbage collection (DGC). Applications using RMI refer to objects in other virtual machines. Garbage cannot be collected in these distributed applications without occasionally collection the local heap, so RMI forces full collections periodically. The frequency of these collections can be controlled with properties. For example,

java -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 ...

specifies explicit collection once per hour instead of the default rate of once per minute. However, this may also cause some objects to take much longer to be reclaimed. These properties can be set as high as Long.MAX_VALUE to make the time between explicit collections effectively infinite, if there is no desire for an upper bound on the timeliness of DGC activity.

Soft References

Soft references are kept alive longer in the server virtual machine than in the client. The rate of clearing can be controlled with the command line option -XX:SoftRefLRUPolicyMSPerMB=<N>, which specifies the number of milliseconds a soft reference will be kept alive (once it is no longer strongly reachable) for each megabyte of free space in the heap. The default value is 1000 ms per megabyte, which means that a soft reference will survive (after the last strong reference to the object has been collected) for 1 second for each megabyte of free space in the heap. Note that this is an approximate figure since soft references are cleared only during garbage collection, which may occur sporadically.

Solaris 8 Alternate libthread

The Solaris 8 Operating System supports an alternate version of the threading library, libthread, that binds threads to light-weight processes (LWPs) directly. Some applications can benefit greatly from the use of this alternate libthread and it is a potential benefit for any threaded application. The following commands will load the alternate libthread for java (Bourne shell syntax is shown):

LD_PRELOAD=/usr/lib/lwp/libthread.so.1
export LD_PRELOAD
java ...

The above is necessary only on Solaris 8, since the alternate libthread is the default in the Solaris 9 Operating System and is the only libthread available starting with Solaris 10.

9. Resources

HotSpot VM Frequently Asked Questions (FAQ)

GC output examples describes how to interpret the output from the different collectors.

How to Handle Java Finalization's Memory-Retention Issues covers finalization pitfalls and ways to avoid them.

Richard Jones and Rafael Lins, Garbage Collection: Algorithms for Automated Dynamic Memory Management, Wiley and Sons (1996), ISBN 0-471-94148-4

As used on the web site, the terms "Java Virtual Machine" and "JVM" mean a virtual machine for the Java platform.

The power of knowledge warehousing

Object class violation when creating group in LDAP using WebSphere Portal

Detailed steps for mapping LDAP attributes in WebSphere Portal 6.1

Messages In Console: Oracle(9581): Floating-Point Assist Fault At Ip [ID 279456.1]

Applies to:

Symptoms

Cause

Solution

References

DBCC CHECKDB

Syntax

Arguments

Remarks

Result Sets

Permissions

Examples

A. Check both the current and the pubs database

B. Check the current database, suppressing informational messages

See Also

DBCC CHECKALLOC (Transact-SQL)

Internal Database Snapshot

Understanding DBCC Error Messages

Error Reporting

Resolving Errors

Reference

Other Resources

Help and Information

NLS Guide for Generic Connectivity and Gateways [ID 230239.1]

How to setup HSODBC on HP-UX [ID 270395.1]

How To Setup Generic Connectivity - HSODBC - on Linux [ID 264567.1]

Tuning Generic Connectivity And Gateways [ID 230543.1]

SQL Server 2005 Books Online (November 2008) DBCC CHECKDB (Transact-SQL)

Internal Database Snapshot

Best Practices

Checking Objects in Parallel

Understanding DBCC Error Messages

Error Reporting

Resolving Errors

Resolving Errors in Database Emergency Mode

Running DBCC CHECKDB with REPAIR_ALLOW_DATA_LOSS in Replicated Databases

A. Checking both the current and the AdventureWorks databases

B. Checking the current database, suppressing informational messages

Reference

Other Resources

Help and Information

How to Configure Generic Connectivity (HSODBC) on Linux 32 bit using DB2Connect [ID 375624.1]

Applies to:

Goal

Solution

References

How To Use The Discoverer 'collectlogs' Utility [ID 259815.1]

Applies to:

Purpose

Last Review Date

Instructions for the Reader

Troubleshooting Details

References

How To Enable Tracing For Discoverer 10g (10.1) Portlet Issues [ID 370337.1]

Applies to:

Goal

Solution

How To Verify Your Discoverer 10g Session (dis51ws) and Preference (dis51pr) Executables Respond Properly [ID 561165.1]

Applies to:

Purpose

Last Review Date

Instructions for the Reader

Troubleshooting Details

How to Relink Oracle Database Software on UNIX [ID 131321.1]

Client Requests Error Out or Time Out (IIS 6.0)

Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning

Table of Contents

Permanent Generation Size

Finalization; Weak, Soft and Phantom References

Explicit Garbage Collection

Soft References

Solaris 8 Alternate libthread