// $Id: JaxpFactory.java,v 1.9 2004/12/15 23:20:29 caron Exp $
/*
 * Copyright 1997-2004 Unidata Program Center/University Corporation for
 * Atmospheric Research, P.O. Box 3000, Boulder, CO 80307,
 * support@unidata.ucar.edu.
 *
 * This library is free software; you can redistribute it and/or modify it
 * under the terms of the GNU Lesser General Public License as published by
 * the Free Software Foundation; either version 2.1 of the License, or (at
 * your option) any later version.
 *
 * This library is distributed in the hope that it will be useful, but
 * WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser
 * General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License
 * along with this library; if not, write to the Free Software Foundation,
 * Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 */

package thredds.catalog;

import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.xml.sax.*;

import java.lang.reflect.*;
import java.io.*;
import java.util.*;

/**
 * Get JAXP DocumentBuilder.
 * Resolve dtd, schemas locally if possible.
 * @author John Caron
 * @version $Id: JaxpFactory.java,v 1.9 2004/12/15 23:20:29 caron Exp $
 */

/*
  PROBLEM: in JAXP 1.2, setting JAXP_SCHEMA_LANGUAGE overrides the DTD
   even when theres no schema declared!

  Cant set JAXP_SCHEMA_LANGUAGE on catalog 0.6, get:
    ** Non-Fatal XML Error error(s) =
    *** XML parser error=cvc-elt.1: Cannot find the declaration of element 'catalog'.
  so validation is not done.

  Of course, you dont know what version until you start parsing !!!!

  "Set Schema Source" in JAXP 1.2: see http://java.sun.com/xml/jaxp/change-requests-11.html

  workaround: make a schema version of old DTD.
  problem: need Xerces parser (crimson wont do schema validation). So if you dont have, we dont even get DTD
    validation.
*/

/*
http://java.sun.com/xml/jaxp/reference/faqs/index.html

Q. How do I use a different JAXP compatible implementation?

The JAXP 1.1 API allows applications to plug in different JAXP compatible implementations of parsers or XSLT processors. For example, when an application wants to create a new JAXP DocumentBuilderFactory instance, it calls the staic method DocumentBuilderFactory.newInstance(). This causes a search for the name of a concrete subclass of DocumentBuilderFactory using the following order:

The value of a system property like javax.xml.parsers.DocumentBuilderFactory if it exists and is accessible.
The contents of the file $JAVA_HOME/jre/lib/jaxp.properties if it exists.
The Jar Service Provider discovery mechanism specified in the Jar File Specification. A jar file can have a resource (i.e. an embedded file) such as META-INF/services/javax.xml.parsers.DocumentBuilderFactory containing the name of the concrete class to instantiate.
The fallback platform default implementation.
Of the above ways to specify an implementation, perhaps the most useful is the jar service provider mechanism. To use this mechanism, place the implementation jar file on your classpath. For example, to use Xerces 1.4.4 instead of the version of Crimson which is bundled with JDK 1.4 (Java Development Kit version 1.4), place xerces.jar in your classpath. This mechanism also works with older versions of the JDK which do not bundle JAXP. If you are using JDK 1.4 and above, see this question for potential problems.

*/

/*
 VERSION notes

  Havent been able to detect how to know what version parser you have. May be new feature in JAXP 1.3

  Java 1.4.2 uses JAXP 1.1, which doesnt do W3S Schema validation.
              (see http://xml.apache.org/~edwingo/jaxp-ri-1.2.0-fcs/docs/)
    when you call factory.setAttribute() get message "No attributes are implemented"
    when you validate=true, get warning "Valid documents must have a <!DOCTYPE declaration.", plus lots of errors,
    but no fatal errors.

  It appears that "Tiger should support the latest version of JAXP...JAXP will be update to conform to the
    new specifications: XML 1.1 Namespaces 1.1 SAX 2.0.1". Note they dont actually say what that is. My
    JavaOne notes claim JAXP 1.3 (JSR 206). Hard to tell what they've added: access to version info and
    perhaps XInclude.

  JAXP 1.2 available in WSDP 1.3.
  Install JAXP 1.2 into J2SE 1.4, see http://java.sun.com/webservices/docs/1.2/jaxp/Updating.html

  "Version 1.4 of the Java 2 platform has a JAXP 1.1 implementation built in. (JAXP 1.1 has a smaller
   application footprint, but JAXP 1.2 implements XML Schema and the transform compiler, XSLTC.)
   Because the built-in libraries are accessed before any classpath entries, you can only access the
   updated libraries by using the Java platform's Endorsed Standards mechanism.
  There are two ways to use the endorsed standards mechanism. The first way is to copy all of the jar
    files except jaxp-api.jar into <JAVA_HOME>/jre/lib/endorsed/
  Note:
   The jaxp-api.jar file should not be copied, because it contains high-level factory APIs that are not
    subject to change.
  Alternatively, you can use the java.endorsed.dirs system property to dynamically add those jar files to
    the JVM when you start your program. Using that system property gives you flexibility of using different
    implementations for different applications.  You specify that property using
     -Djava.endorsed.dirs=yourDirectoryPath.
  Finally, although the endorsed directory mechanism guarantees that the specified jars will be searched
   before the internal classes in the VM (which are searched before the classpath), the order in which the
   jar files are searched is indeterminate. For that reason, there should be no overlaps in the classes
   specified using that property. For more information, see the Endorsed Standards documentation for the
   1.4 version of the Java platform."

  Webstart see http://forum.java.sun.com/thread.jsp?forum=38&thread=231995

"With my workaround for the context classloader in-place (see bug report) all I've ever needed to switch
 between xerces and crimson is the single property;
 <property name="javax.xml.parsers.SAXParserFactory" value="org.apache.xerces.jaxp.SAXParserFactoryImpl"/>

added into my jnlp; along with the jars, jaxp.jar, xercesImpl.jar & xmlParserAPIs.jar. (jnlp also needs to
   work under 1.2 & 1.3 - and it does!).
Hardcoding the parser just defeats the purpose of JAXP, although I too am wondering just which parser is
  going to be the default in the future - I was expecting xerces in 1.4.1 as it's the only way to handle
  schemas, maybe when JAXP gets updated with its schema properties/features I guess. "

----------

Add these to the jnlp file: (see Viewer.jnlp):

 <property name="javax.xml.parsers.SAXParserFactory" value="org.apache.xerces.jaxp.SAXParserFactoryImpl"/>
 <property name="javax.xml.parsers.DocumentBuilderFactory" value="org.apache.xerces.jaxp.DocumentBuilderFactoryImpl"/>

 these 2 classes are in xercesImpl.jar

  org.apache.xerces.* not in 1.4.2/rt.jar
*/

/* 4/26/04
   not apparently picking up the unique/key/keyref constraints in schema.
   if catalog has 2 datasets with same ID; dont get an error.
   does give an error using catalog 0.6, which still uses schema, but uses xs:ID type.
 */

/* 5/10/04
  tomcat (5.0.19, 5.0.24) ships with "Xerces-J 2.6.1" in common/endorsed
*/

public class JaxpFactory {
  static private boolean debugEntityResolution = false, debugFactory = false, debugMessages = false;
  static private DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
  static private HashMap entityHash = new HashMap();

  // schema validation
  static private boolean schemaValidationOk = true;
  static private final String JAXP_SCHEMA_LANGUAGE = "http://java.sun.com/xml/jaxp/properties/schemaLanguage";
  static public final String W3C_XML_NAMESPACE = "http://www.w3.org/2001/XMLSchema";
  static private final String JAXP_SCHEMA_SOURCE = "http://java.sun.com/xml/jaxp/properties/schemaSource";

  // catalog namespaces
  static public final String CATALOG_NAMESPACE_06 = "http://www.unidata.ucar.edu/thredds";
  static public final String CATALOG_NAMESPACE_10 = "http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
  // dqc namespaces
  static public final String DQC_NAMESPACE_02 = "http://www.unidata.ucar.edu/schemas/thredds/queryCapability";
  static public final String DQC_NAMESPACE_03 = "http://www.unidata.ucar.edu/namespaces/thredds/queryCapability/v0.3";

  // nj22 namespaces
  static public final String NJ22_NAMESPACE = "http://www.unidata.ucar.edu/namespaces/netcdf/nj22";

  // other namespaces
  static public final String XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";

  static {
    // Get Document Builder Factory
    docBuilderFactory = DocumentBuilderFactory.newInstance();
    docBuilderFactory.setNamespaceAware(true);

    // try to get schema validation
    if (schemaValidationOk) {
      try {
        docBuilderFactory.setAttribute(JAXP_SCHEMA_LANGUAGE, W3C_XML_NAMESPACE);
        docBuilderFactory.setAttribute(JAXP_SCHEMA_SOURCE,
           new String[] { CATALOG_NAMESPACE_06, CATALOG_NAMESPACE_10, XLINK_NAMESPACE, DQC_NAMESPACE_02, DQC_NAMESPACE_03 } );
      }
      catch (IllegalArgumentException e) {
        System.out.println("***Warning: XML Parser does not support JAXP 1.2. Library will continue without validation.\n"+e);
        schemaValidationOk = false;
      }
    }

    if (debugFactory) showFactoryInfo( docBuilderFactory);

    // set up entity resolution - see MyEntityResolver class
    // catalog 0.6 dtd
    initEntity( "http://www.unidata.ucar.edu/projects/THREDDS/xml/InvCatalog.0.6.dtd",
                "/resources/xml/InvCatalog.0.6.dtd",
                "http://www.unidata.ucar.edu/projects/THREDDS/xml/InvCatalog.0.6.dtd");

    // catalog 1.0 schema
    initEntity( CATALOG_NAMESPACE_10,
                "/resources/xml/InvCatalog.1.0.xsd",
                "http://www.unidata.ucar.edu/schemas/thredds/InvCatalog.1.0.xsd");

    // catalog 0.6 schema
    initEntity( CATALOG_NAMESPACE_06,
                "/resources/xml/InvCatalog.0.6.xsd",
                "http://www.unidata.ucar.edu/schemas/thredds/InvCatalog.0.6.xsd");

    // DQC schema 0.2
    initEntity( DQC_NAMESPACE_02,
                "/resources/xml/queryCapability.0.2.xsd",
                "http://www.unidata.ucar.edu/schemas/thredds/queryCapability.0.2.xsd");

    // DQC schema 0.3
    initEntity( DQC_NAMESPACE_03,
                "/resources/xml/queryCapability.0.3.xsd",
                "http://www.unidata.ucar.edu/schemas/thredds/queryCapability.0.3.xsd");

    // nj22 schema
    initEntity( NJ22_NAMESPACE,
                "/resources/xml/ncml-2.2.xsd",
                "http://www.unidata.ucar.edu/schemas/netcdf/ncml-2.2.xsd");

    // XLink schema
    initEntity( XLINK_NAMESPACE,
                "/resources/xml/xlink.xsd",
                "http://www.unidata.ucar.edu/schemas/other/xlink.xsd");

  }

  static private void initEntity( String entityName, String resourceName, String urlName) {
    String entity = null;

    try { // try to read from local file resource, eg from catalog.jar
      ByteArrayOutputStream sbuff = new ByteArrayOutputStream(3000);
      InputStream is = thredds.util.Resource.getFileResource( resourceName);
      if (is != null) {
        thredds.util.IO.copy(is, sbuff);
        entity = sbuff.toString();
        if (debugEntityResolution) System.out.println(" *** entity "+entityName+" mapped to local resource at "+resourceName);

      } else if (urlName != null) { // otherwise, get from network
        entity = thredds.util.IO.readURLcontentsWithException(urlName);
        if (debugEntityResolution) System.out.println(" *** entity "+entityName+" mapped to remote URL at "+urlName);
      }

    } catch (IOException e) {
      System.out.println(" *** FAILED to map entity "+entityName+" locally at "+resourceName+" or remotely at "+urlName);
      // e.printStackTrace();
    }

    entityHash.put( entityName, entity);
    entityHash.put( urlName, entity); // also map it to the remote URL
  }

  static private void showFactoryInfo(DocumentBuilderFactory factory) {
    System.out.println("------------------------");
    System.out.println("DocumentBuilderFactory class= "+factory.getClass().getName());
    try {
      //ClassLoader cl = factory.getClass().getClassLoader();
      //Class version = cl.loadClass("org.apache.xerces.impl.Version");
      Class version = Class.forName("org.apache.xerces.impl.Version");
      Method m = version.getMethod("getVersion", null);
      System.out.println(" org.apache.xerces.impl.Version.version()="+m.invoke(null, null));

      thredds.util.reflect.ProxyGenerator.showMethods( factory.getClass(), System.out);
    } catch (Exception e) {
      e.printStackTrace();
    }
    System.out.println();

    System.out.println(" factory.isValidating()="+factory.isValidating());
    System.out.println(" factory.isNamespaceAware()="+factory.isNamespaceAware());
    System.out.println(" factory.isIgnoringElementContentWhitespace()="+factory.isIgnoringElementContentWhitespace());
    System.out.println(" factory.isExpandEntityReferences()="+factory.isExpandEntityReferences());
    System.out.println(" factory.isIgnoringComments()="+factory.isIgnoringComments());
    System.out.println(" factory.isCoalescing()="+factory.isCoalescing());

  }

  static private void showBuilderInfo( DocumentBuilder builder) {
    System.out.println("-----------------------");
    System.out.println(" builder.isValidating()="+builder.isValidating());
    System.out.println(" builder.isNamespaceAware()="+builder.isNamespaceAware());

    System.out.println("DocumentBuilder class= "+builder.getClass().getName());
    try {
      thredds.util.reflect.ProxyGenerator.showMethods( builder.getClass(), System.out);
      thredds.util.reflect.ProxyGenerator.showMethods( builder.getDOMImplementation().getClass(), System.out);
    } catch (Exception e) {
      e.printStackTrace();
    }

  }

  ////////////////////////////////////////////////////////////////////////////////////////////////
  private DocumentBuilder builder; // JAXP parser

  private HashMap versionHash = new HashMap(10);
  private StringBuffer warnMessages = new StringBuffer();
  private StringBuffer errMessages = new StringBuffer();
  private StringBuffer fatalMessages = new StringBuffer();
  private String version = "";

  /** Constructor.
   * @param validate : do XML validation or not.
   */
  public JaxpFactory(boolean validate) {


    try {
      synchronized (docBuilderFactory) { // Only use one DocumentBuilderFactory, so needs to be synchronized
          docBuilderFactory.setValidating(validate && schemaValidationOk);
          builder = docBuilderFactory.newDocumentBuilder();
      }
      builder.setEntityResolver(new MyEntityResolver());
      builder.setErrorHandler(new MyErrorHandler());

      if (debugFactory) showBuilderInfo(builder);

    } catch (ParserConfigurationException e) {
      System.out.println("The underlying parser does not support the requested features.");
      throw new UnsupportedOperationException();

    } catch (FactoryConfigurationError e) {
      System.out.println("Error occurred obtaining Document Builder Factory.");
      throw new UnsupportedOperationException();
    }

  }

  public DocumentBuilder getDocumentBuilder() { return builder; }
  public StringBuffer getWarningMessages() { return warnMessages; }
  public StringBuffer getErrorMessages() { return errMessages; }
  public StringBuffer getFatalMessages() { return fatalMessages; }
  public String getVersion() { return version; }

  // helper classes for JAXP parsing
  private class MyEntityResolver implements org.xml.sax.EntityResolver {

    // we read the DTD locally if we can.
    public org.xml.sax.InputSource resolveEntity(String publicId, String systemId) throws SAXException, IOException {
      if (debugEntityResolution) System.out.print("  publicId="+publicId+" systemId="+systemId);
      String entity = (String) entityHash.get( systemId);
      if (entity != null) {
        if (debugEntityResolution) System.out.println(" *** resolved  with local copy");
        return new MyInputSource(entity);
      }

      if (systemId.indexOf("InvCatalog.0.6.dtd") >= 0) {
        entity = (String) entityHash.get( "http://www.unidata.ucar.edu/projects/THREDDS/xml/InvCatalog.0.6.dtd");
        if (entity != null) {
          if (debugEntityResolution) System.out.println(" *** resolved2 with local copy");
          return new MyInputSource(entity);
        }
      }

      if (debugEntityResolution) System.out.println(" *** not resolved");
      return null;
    }

  }

  private class MyInputSource extends org.xml.sax.InputSource {
    MyInputSource( String entity) {
      setCharacterStream(new StringReader(entity));

    }
  }

  private class MyErrorHandler implements org.xml.sax.ErrorHandler {

    public void warning(SAXParseException e) throws SAXException {
      warnMessages.append("*** XML parser warning="+e.getMessage()+"\n");
      if (debugMessages) System.out.println("*** XML parser warning="+e.getMessage()+"\n");
    }

    public void error(SAXParseException e) throws SAXException {
      errMessages.append("*** XML parser error="+e.getMessage()+"\n");
      if (debugMessages) System.out.println("*** XML parser error="+e.getMessage()+"\n");
    }

    public void fatalError(SAXParseException e) throws SAXException {
      fatalMessages.append("*** XML parser fatalError="+e.getMessage()+"\n");
      if (debugMessages) System.out.println("*** XML parser fatalError="+e.getMessage()+"\n");
    }

  }


  /************************************************************************/
  // test

  private static void doOne( JaxpFactory fac, DocumentBuilder builder, String url) {
    System.out.println("***read "+url);
    fac.getFatalMessages().setLength(0);
    fac.getErrorMessages().setLength(0);
    try {
      Document doc = builder.parse( url);
      System.out.println(" fatal= " + fac.getFatalMessages());
      System.out.println(" errors= " + fac.getErrorMessages());
      System.out.println(" warn= " + fac.getWarningMessages());
    } catch (Exception e) { e.printStackTrace(); }

  }
   /** testing */
  public static void main (String[] args) throws Exception {
    JaxpFactory fac = new JaxpFactory(true);
    DocumentBuilder builder = fac.getDocumentBuilder();

    /* cat 0.6
    doOne( fac, builder, "file:///C:/dev/thredds/catalog/test/data/InvCatalog.0.6.xml");
    doOne( fac, builder, "file:///C:/dev/thredds/catalog/test/data/ParseFails.xml");
    doOne( fac, builder, "file:///C:/dev/thredds/catalog/test/data/MalFormed.xml");

    // cat 1.0 */
    doOne( fac, builder, "file:///C:/dev/thredds/catalog/test/data/Example1.0rc7.xml"); // */

    /* dqc 0.2
    doOne( fac, builder, "file:///C:/dev/thredds/catalog/test/data/dqc/zoneDQC.xml");

    // dqc 0.3
    doOne( fac, builder, "file:///C:/dev/thredds/catalog/test/data/dqc/exampleDqc.xml"); // */
  }

}

/* Change History:
   $Log: JaxpFactory.java,v $
   Revision 1.9  2004/12/15 23:20:29  caron
   NJ22 namespace

   Revision 1.8  2004/12/15 23:09:32  edavis
   Update NJ22_NAMESPACE value.

   Revision 1.7  2004/11/30 21:44:54  edavis
   Synchronize call to docBuilderFactory.setValidating().

   Revision 1.6  2004/11/07 03:00:47  caron
   *** empty log message ***

   Revision 1.5  2004/10/06 19:03:39  caron
   clean up javadoc
   change useV3 -> useRecordsAsStructure
   remove id, title, from NetcdfFile constructors
   add "in memory" NetcdfFile

   Revision 1.4  2004/09/28 21:39:09  caron
   *** empty log message ***

   Revision 1.3  2004/09/24 03:26:27  caron
   merge nj22

   Revision 1.2  2004/06/04 00:51:54  caron
   release 2.0b

   Revision 1.1  2004/05/11 23:30:28  caron
   release 2.0a


 */