Custom autoComplete component

curiousgally · November 5, 2011, 5:42am

I’m trying to create a custom auto complete component using a text item and list grid as shown in the image attached. It is supposed to function as ComboBoxItem. But I wanted to add additional functionalities like multiple select etc which are missing in ComboBoxItem. Hence thought of using a listGrid to show the picklist in ComboboxItem. Any help on ways to handle keyboard navigation i.e. user should be able to navigate from textitem to listgrid with up and down arrows. Also I need to hide the list grid when clicked outside textitem and listgrid.

Thanks.

curiousgally · November 5, 2011, 10:33am

To be more specific, I’m looking for an approach to achieve what happens when a menu is opened and clicked outside. It closes, similarly I want listgrid to hide in this case.

Thanks.

vic_123 · November 5, 2011, 1:41pm

very easy…
Use pop up on click of formitemicon assciated with textitem…

curiousgally · November 5, 2011, 2:23pm

Hi,

Thanks for the reply. As I mentioned I don’t want to use a picker icon. I"m trying to use the listgrid and provide auto complete feature by filtering the list grid as and when the user types in the textitem accordingly. I got the filter to work appropriately but need only to implement the keyboard navigation and show/hide implementation between textitem and listgrid. So I need to hide the listgrid when clicked outside the components textitem and listgrid but on the page. Hope you understand what i’m trying to achieve.

Thanks.

curiousgally · November 6, 2011, 4:33pm

I could able to achieve this behavior. Its achieved using showClickMask method on the canvas. It took me some time though to understand how it works. I feel this is very essential when developing custom components. Strangely I didn’t find a single example or explanation anywhere on the internet about using Masks other than the method description in the Canvas API class.

Isomorphic, I would suggest you to add this usecase into the existing Custom picklist sample in the showcase.

Also I have a question,
showClickMask method takes only array of Canvas items but in my case I would be glad if this works for both TextItem and ListGrid. Any way of achieving this. I’m creating a component with a formitem and a associated listgrid.

Thanks.

curiousgally · November 8, 2011, 4:23am

Isomorphic,

I will be using this custom component to auto complete airport, cities, countries etc. The table size is having 13000 records and changes to the table are bare minimal. I will be having multiple form components using this datasource. What would be the best approach to load this data at once and re-use it for multiple components.

I used this for a single component(textitem + listgrid as in the image attached in the previous post) and tried with paging(without ‘LOCAL’) data mode it was lagging a lot.
I went ahead with ‘LOCAL’ data mode and i see little bit of lagging but it is lot better than the previous case.
With Paging MODE (Without ‘LOCAL’ mode) i set the setDataPageSize(15000). The data comes back in 7 seconds. But when I use ‘LOCAL’ mode or do a direct fetch data on the DataSource itself it takes more than 6 minutes to fetch the entire data.
If it can be retrieved through datasource in less than 7 seconds, I would like to know how to initiate these in the background so that as soon as application welcome page is shown, the datasource can load the data.

Thanks.

Isomorphic · November 8, 2011, 5:19pm

Don’t try to load this many records into the browser if you need to support IE6. Even if you only need to support IE7 and IE8, we’d still recommend against it. Most applications will not perform enough operations against a 13000 row dataset to justify the database load of grabbing the entire dataset up front.

However if you’re in the rare case where you only to support one browser and it’s Chrome or Firefox, you can load the whole dataset up front by setting clientOnly:true on the DataSource. Once you’ve set clientOnly:true, the first data access will trigger the whole dataset to be loaded - you can either allow a component to do this, or call DataSource.fetchData() directly.

curiousgally · November 9, 2011, 12:26am

A very minimal transaction is taking a minimum of four seconds which is not acceptable for a autocomplete feature while it shouldnt be considering the table size and the database(oracle). In our earlier j2ee products we have done this using server side jcs cache which queries only once for the whole data set when application is started. This is something which many components in application would be using. Also I have mentioned in my earlier post that it takes more time to do a fetch on data source (6mints) compared to setting pagesize and fetch on listgrid(7sec) . How to fetch it with same speed using data source .

Isomorphic · November 9, 2011, 12:33am

You seem to have an unbelievably slow database - that’s not normal even for 13,000 records. In fact it’s not normal by about 100x.

Usually when someone reports this kind of slowness it is the SQL query itself that is taking forever to execute because it, for example, is accessing a database view that does a complex join query across many tables. This is easy to figure out using the server-side log, which includes timestamps.

As far as DataSource vs ListGrid speed, most likely, your DataSource.fetchData() call is not specifying dsRequest.startRow and endRow so it is loading the whole dataset, whereas the ListGrid automatically sets startRow and endRow to load only the initially visible data.

curiousgally · November 9, 2011, 1:49am

Hi,

I’m setting the datapagesize(15000) on listgrid while table has 13000 rows only , which means it pulls the whole data in the table. It come to the client in less than 7 seconds whereas a pull from datasource directly takes 6 mints. Also data fetch by listgrid without using paging and using dataFetchMode(LOCAL) takes 6 minutes too.

Thanks.

Isomorphic · November 9, 2011, 3:13am

Yeah that makes no sense. Looks at the server log timestamps as previously suggested, and also atthe turnaround times reported in the RPC tab, to understand where the time is being spent. If you need help, post complete information.

curiousgally · November 9, 2011, 6:14am

Hi,

Please find the log timestampls below:

=== 2011-11-09 11:16:09,465 [81-4] INFO  RequestContext - URL: '/mda/builtinds/sc/skins/Enterprise/images/Scrollbar/hthumb_start.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0': Moz (Gecko) with Accept-Encoding header
=== 2011-11-09 11:16:09,477 [81-1] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"bo"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"bo"
            }
        ],
        _constructor:"AdvancedCriteria"
    },
    operationConfig:{
        dataSource:"typeAhead",
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    startRow:0,
    endRow:75,
    componentId:"isc_ListGrid_0",
    appID:"builtinApplication",
    operation:"typeAhead_fetch",
    oldValues:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"bo"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"bo"
            }
        ],
        _constructor:"AdvancedCriteria"
    }
}
=== 2011-11-09 11:16:09,478 [81-2] INFO  RequestContext - URL: '/mda/builtinds/sc/skins/Enterprise/images/Scrollbar/hthumb_end.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0': Moz (Gecko) with Accept-Encoding header
=== 2011-11-09 11:16:09,518 [81-2] INFO  Download - done streaming: C:/Users/sg0209257/HarshaGalla/CodeBase/jars/smartgwtee-3.0/samples/built-in-ds/war/builtinds/sc/skins/Enterprise/images/Scrollbar/hthumb_end.png
=== 2011-11-09 11:16:09,532 [81-1] INFO  IDACall - Performing 1 operation(s)
=== 2011-11-09 11:16:09,593 [81-1] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2011-11-09 11:16:09,593 [81-1] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No public zero-argument method named '_typeAhead_fetch' found, performing generic datasource operation
=== 2011-11-09 11:16:09,594 [81-1] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] Performing fetch operation with
	criteria: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"bo"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"bo"}],operator:"or",_constructor:"AdvancedCriteria"}	values: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"bo"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"bo"}],operator:"or",_constructor:"AdvancedCriteria"}
=== 2011-11-09 11:16:09,610 [81-1] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
log4j:WARN No appenders could be found for logger (org.apache.Velocity).
log4j:WARN Please initialize the log4j system properly.
=== 2011-11-09 11:16:10,031 [81-1] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Executing row count query: SELECT COUNT(*) FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:16:10,031 [81-1] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Eval'd row count query: SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('bo%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%bo%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:16:10,212 [81-1] DEBUG PoolableSQLConnectionFactory - [builtinApplication.typeAhead_fetch] Initializing SQL config for 'Oracle' from system config - using DataSource:  oracle.jdbc.pool.OracleDataSource
=== 2011-11-09 11:16:13,281 [81-1] DEBUG PoolableSQLConnectionFactory - [builtinApplication.typeAhead_fetch] Returning pooled Connection
=== 2011-11-09 11:16:13,955 [81-1] INFO  SQLDriver - [builtinApplication.typeAhead_fetch] Executing SQL query on 'Oracle': SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('bo%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%bo%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:16:14,597 [81-1] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] JDBC driver windowed select rows 0->75, result size 75. Query: SELECT AIRPORT_IATA_CODE, AIRPORT_NAME, COUNTRY_NAME FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('bo%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%bo%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:16:15,281 [81-1] INFO  DSResponse - [builtinApplication.typeAhead_fetch] DSResponse: List with 75 items
=== 2011-11-09 11:16:15,597 [81-1] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2011-11-09 11:16:15,598 [81-1] DEBUG RPCManager - DMI response, dropExtraFields: true

=== 2011-11-09 11:16:48,907 [81-6] INFO  RequestContext - URL: '/mda/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.106 Safari/535.2': Safari with Accept-Encoding header
=== 2011-11-09 11:16:48,910 [81-6] DEBUG XML - Parsed XML from (in memory stream): 2ms
=== 2011-11-09 11:16:48,915 [81-6] DEBUG RPCManager - Processing 1 requests.
=== 2011-11-09 11:16:48,918 [81-6] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"amd"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"amd"
            }
        ],
        _constructor:"AdvancedCriteria"
    },
    operationConfig:{
        dataSource:"typeAhead",
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    startRow:0,
    endRow:75,
    componentId:"isc_ListGrid_0",
    appID:"builtinApplication",
    operation:"typeAhead_fetch",
    oldValues:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"amd"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"amd"
            }
        ],
        _constructor:"AdvancedCriteria"
    }
}
=== 2011-11-09 11:16:48,919 [81-6] INFO  IDACall - Performing 1 operation(s)
=== 2011-11-09 11:16:48,920 [81-6] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2011-11-09 11:16:48,920 [81-6] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No public zero-argument method named '_typeAhead_fetch' found, performing generic datasource operation
=== 2011-11-09 11:16:48,921 [81-6] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] Performing fetch operation with
	criteria: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"amd"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"amd"}],operator:"or",_constructor:"AdvancedCriteria"}	values: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"amd"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"amd"}],operator:"or",_constructor:"AdvancedCriteria"}
=== 2011-11-09 11:16:48,921 [81-6] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:16:48,922 [81-6] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Executing row count query: SELECT COUNT(*) FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:16:48,922 [81-6] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Eval'd row count query: SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('amd%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%amd%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))

=== 2011-11-09 11:16:51,529 [81-6] DEBUG PoolableSQLConnectionFactory - [builtinApplication.typeAhead_fetch] Returning pooled Connection
=== 2011-11-09 11:16:52,094 [81-6] INFO  SQLDriver - [builtinApplication.typeAhead_fetch] Executing SQL query on 'Oracle': SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('amd%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%amd%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:16:52,686 [81-6] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] JDBC driver windowed select rows 0->75, result size 75. Query: SELECT AIRPORT_IATA_CODE, AIRPORT_NAME, COUNTRY_NAME FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('amd%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%amd%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:16:53,274 [81-6] INFO  DSResponse - [builtinApplication.typeAhead_fetch] DSResponse: List with 6 items
=== 2011-11-09 11:16:53,563 [81-6] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2011-11-09 11:16:53,564 [81-6] DEBUG RPCManager - DMI response, dropExtraFields: true
=== 2011-11-09 11:16:53,593 [81-6] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\DynamicForm\unchecked.png
=== 2011-11-09 11:16:53,656 [81-6] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Scrollbar\hscroll_Disabled_start.png
=== 2011-11-09 11:16:53,660 [81-3] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Scrollbar\hscroll_Disabled_end.png
=== 2011-11-09 11:16:53,664 [81-1] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Scrollbar\hscroll_Disabled_track.png
=== 2011-11-09 11:16:59,816 [81-1] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\ListGrid\row_Over.png
=== 2011-11-09 11:17:00,653 [81-1] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\ListGrid\row_Over_Selected.png
=== 2011-11-09 11:17:00,656 [81-3] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\DynamicForm\checked.png
=== 2011-11-09 11:17:02,958 [81-3] INFO  RequestContext - URL: '/mda/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.106 Safari/535.2': Safari with Accept-Encoding header
=== 2011-11-09 11:17:02,961 [81-3] DEBUG XML - Parsed XML from (in memory stream): 2ms
=== 2011-11-09 11:17:02,966 [81-3] DEBUG RPCManager - Processing 1 requests.
=== 2011-11-09 11:17:02,967 [81-3] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"che"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"che"
            }
        ],
        _constructor:"AdvancedCriteria"
    },
    operationConfig:{
        dataSource:"typeAhead",
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    startRow:0,
    endRow:75,
    componentId:"isc_ListGrid_0",
    appID:"builtinApplication",
    operation:"typeAhead_fetch",
    oldValues:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"che"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"che"
            }
        ],
        _constructor:"AdvancedCriteria"
    }
}
=== 2011-11-09 11:17:02,967 [81-3] INFO  IDACall - Performing 1 operation(s)
=== 2011-11-09 11:17:02,968 [81-3] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2011-11-09 11:17:02,968 [81-3] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No public zero-argument method named '_typeAhead_fetch' found, performing generic datasource operation
=== 2011-11-09 11:17:02,968 [81-3] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] Performing fetch operation with
	criteria: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"che"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"che"}],operator:"or",_constructor:"AdvancedCriteria"}	values: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"che"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"che"}],operator:"or",_constructor:"AdvancedCriteria"}
=== 2011-11-09 11:17:02,973 [81-3] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:17:02,974 [81-3] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Executing row count query: SELECT COUNT(*) FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:17:02,974 [81-3] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Eval'd row count query: SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('che%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%che%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:17:05,799 [81-3] DEBUG PoolableSQLConnectionFactory - [builtinApplication.typeAhead_fetch] Returning pooled Connection
=== 2011-11-09 11:17:06,362 [81-3] INFO  SQLDriver - [builtinApplication.typeAhead_fetch] Executing SQL query on 'Oracle': SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('che%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%che%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:17:06,949 [81-3] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] JDBC driver windowed select rows 0->75, result size 75. Query: SELECT AIRPORT_IATA_CODE, AIRPORT_NAME, COUNTRY_NAME FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('che%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%che%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:17:07,537 [81-3] INFO  DSResponse - [builtinApplication.typeAhead_fetch] DSResponse: List with 75 items
=== 2011-11-09 11:17:07,822 [81-3] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2011-11-09 11:17:07,823 [81-3] DEBUG RPCManager - DMI response, dropExtraFields: true

=== 2011-11-09 11:17:10,995 [81-5] INFO  RequestContext - URL: '/mda/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.106 Safari/535.2': Safari with Accept-Encoding header
=== 2011-11-09 11:17:10,999 [81-5] DEBUG XML - Parsed XML from (in memory stream): 3ms
=== 2011-11-09 11:17:11,006 [81-5] DEBUG RPCManager - Processing 1 requests.
=== 2011-11-09 11:17:11,008 [81-5] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"dallas"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"dallas"
            }
        ],
        _constructor:"AdvancedCriteria"
    },
    operationConfig:{
        dataSource:"typeAhead",
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    startRow:0,
    endRow:75,
    componentId:"isc_ListGrid_0",
    appID:"builtinApplication",
    operation:"typeAhead_fetch",
    oldValues:{
        operator:"or",
        criteria:[
            {
                fieldName:"AIRPORT_IATA_CODE",
                operator:"iStartsWith",
                value:"dallas"
            },
            {
                fieldName:"AIRPORT_NAME",
                operator:"iContains",
                value:"dallas"
            }
        ],
        _constructor:"AdvancedCriteria"
    }
}
=== 2011-11-09 11:17:11,008 [81-5] INFO  IDACall - Performing 1 operation(s)
=== 2011-11-09 11:17:11,009 [81-5] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2011-11-09 11:17:11,010 [81-5] DEBUG AppBase - [builtinApplication.typeAhead_fetch] No public zero-argument method named '_typeAhead_fetch' found, performing generic datasource operation
=== 2011-11-09 11:17:11,010 [81-5] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] Performing fetch operation with
	criteria: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"dallas"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"dallas"}],operator:"or",_constructor:"AdvancedCriteria"}	values: {criteria:[{fieldName:"AIRPORT_IATA_CODE",operator:"iStartsWith",value:"dallas"},{fieldName:"AIRPORT_NAME",operator:"iContains",value:"dallas"}],operator:"or",_constructor:"AdvancedCriteria"}
=== 2011-11-09 11:17:11,011 [81-5] INFO  SQLDataSource - [builtinApplication.typeAhead_fetch] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:17:11,011 [81-5] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Executing row count query: SELECT COUNT(*) FROM $defaultTableClause WHERE $defaultWhereClause
=== 2011-11-09 11:17:11,027 [81-5] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] Eval'd row count query: SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('dallas%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%dallas%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:17:11,029 [81-1] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Scrollbar\vscroll_Disabled_end.png
=== 2011-11-09 11:17:11,036 [81-6] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Scrollbar\vscroll_Disabled_track.png
=== 2011-11-09 11:17:11,030 [81-2] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Scrollbar\vscroll_Disabled_start.png
=== 2011-11-09 11:17:13,604 [81-5] DEBUG PoolableSQLConnectionFactory - [builtinApplication.typeAhead_fetch] Returning pooled Connection
=== 2011-11-09 11:17:14,165 [81-5] INFO  SQLDriver - [builtinApplication.typeAhead_fetch] Executing SQL query on 'Oracle': SELECT COUNT(*) FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('dallas%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%dallas%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:17:14,749 [81-5] DEBUG SQLDataSource - [builtinApplication.typeAhead_fetch] JDBC driver windowed select rows 0->75, result size 75. Query: SELECT AIRPORT_IATA_CODE, AIRPORT_NAME, COUNTRY_NAME FROM AIRPORT_DIM WHERE ((LOWER(AIRPORT_IATA_CODE) LIKE LOWER('dallas%') ESCAPE'\'  AND AIRPORT_IATA_CODE IS NOT NULL) OR (LOWER(AIRPORT_NAME) LIKE LOWER('%dallas%') ESCAPE'\'  AND AIRPORT_NAME IS NOT NULL))
=== 2011-11-09 11:17:15,331 [81-5] INFO  DSResponse - [builtinApplication.typeAhead_fetch] DSResponse: List with 11 items
=== 2011-11-09 11:17:15,617 [81-5] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2011-11-09 11:17:15,617 [81-5] DEBUG RPCManager - DMI response, dropExtraFields: true
=== 2011-11-09 11:17:23,857 [81-5] INFO  RequestContext - URL: '/mda/builtinds/sc/skins/Enterprise/images/Scrollbar/vscroll_Over_start.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.106 Safari/535.2': Safari with Accept-Encoding header
=== 2011-11-09 11:20:26,753 [81-6] INFO  RequestContext - URL: '/mda/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.106 Safari/535.2': Safari with Accept-Encoding header
=== 2011-11-09 11:20:26,756 [81-6] DEBUG XML - Parsed XML from (in memory stream): 2ms
=== 2011-11-09 11:20:26,761 [81-6] DEBUG RPCManager - Processing 1 requests.
=== 2011-11-09 11:20:26,761 [81-6] DEBUG RPCManager - Request #1 (RPCRequest) data: {
    appID:"isc_builtin",
    className:"builtin",
    methodName:"getAvailableScriptEngines",
    arguments:[
    ],
    is_ISC_RPC_DMI:true
}
=== 2011-11-09 11:20:26,762 [81-6] INFO  IDACall - Performing 1 operation(s)
=== 2011-11-09 11:20:26,807 [81-1] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\cssButton\button_stretch.png
=== 2011-11-09 11:20:26,810 [81-5] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\sg0209257\HarshaGalla\CodeBase\jars\smartgwtee-3.0\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\Tab\top\tab_Disabled_start.png

I have removed some log entries related to images due to character constraints. Also find the attached screen shot.

Find the code below:

public class TypeAheadTextItem extends TextItem {
    private ListGrid listGrid;

    public TypeAheadTextItem() {
        super();
        listGrid = new ListGrid();
        listGrid.setDataSource(DataSource.get("typeAhead"));
        ListGridField airportId = new ListGridField("AIRPORT_IATA_CODE",75);
        ListGridField airportName = new ListGridField("AIRPORT_NAME",250);
        listGrid.setFields(airportId, airportName);
        listGrid.setShowHeader(false);
        listGrid.setSelectionAppearance(SelectionAppearance.CHECKBOX);
        listGrid.setCanSelectAll(false);
        listGrid.setAutoFitData(Autofit.BOTH);
//        listGrid.setDataPageSize(15000);
        listGrid.setAutoFitMaxRecords(10);
//        listGrid.fetchData();
        listGrid.addDataArrivedHandler(new DataArrivedHandler() {
            @Override
            public void onDataArrived(DataArrivedEvent dataArrivedEvent) {
                RecordList recordList= listGrid.getDataAsRecordList();
//                String searchText = getEnteredValue().contains(",") ? getEnteredValue().substring(getEnteredValue().lastIndexOf(",") + 1).trim() : getEnteredValue();
                String[] selectedItems = getEnteredValue().contains(",")?getEnteredValue().substring(0,getEnteredValue().lastIndexOf(",")).split(","):new String[]{};
                for(String item:selectedItems){
                    if(recordList.find("AIRPORT_IATA_CODE",item)!=null)
                        listGrid.selectRecord(recordList.find("AIRPORT_IATA_CODE",item));
                }
            }
        });
        listGrid.addRecordClickHandler(new RecordClickHandler() {
            @Override
            public void onRecordClick(RecordClickEvent recordClickEvent) {
                Record rec = recordClickEvent.getRecord();
                String value = getEnteredValue().contains(",") ? getEnteredValue().substring(0, getEnteredValue().lastIndexOf(",") + 1).trim() : "";
                setValue(value + rec.getAttribute("AIRPORT_IATA_CODE") + ",");
                focusInItem();
            }
        });


        addKeyUpHandler(new KeyUpHandler() {
            @Override
            public void onKeyUp(KeyUpEvent keyUpEvent) {
                if (keyUpEvent.getKeyName().equalsIgnoreCase("escape"))
                    listGrid.hide();
            }
        });

        addChangedHandler(new ChangedHandler() {
            @Override
            public void onChanged(ChangedEvent changedEvent) {
                String searchText = getEnteredValue().contains(",") ? getEnteredValue().substring(getEnteredValue().lastIndexOf(",") + 1).trim() : getEnteredValue().trim();
                if (!searchText.equalsIgnoreCase("")) {
                    new Timer() {
                        private String currentValue = getEnteredValue().contains(",") ? getEnteredValue().substring(getEnteredValue().lastIndexOf(",") + 1).trim() : getEnteredValue();

                        @Override
                        public void run() {
                            String searchText = getEnteredValue().contains(",") ? getEnteredValue().substring(getEnteredValue().lastIndexOf(",") + 1).trim() : getEnteredValue();
                            if (currentValue.equalsIgnoreCase(searchText)) {
                                AdvancedCriteria criteria = new AdvancedCriteria(OperatorId.OR, new Criterion[]{
                                        new Criterion("AIRPORT_IATA_CODE", OperatorId.ISTARTS_WITH, searchText),
                                        new Criterion("AIRPORT_NAME", OperatorId.ICONTAINS, searchText)});
                                listGrid.fetchData(criteria);
                                listGrid.show();
                                listGrid.moveTo(getLeft(), getTop() + getHeight());
                                listGrid.showClickMask(new ListGridMask(listGrid), ClickMaskMode.HARD, new Canvas[]{listGrid});
                                focusInItem();
                            }
                        }
                    }.schedule(400);
                } else listGrid.hide();
            }
        });
    }
}

I have set 400ms between key events to avoid hitting server. But if you see in the RPC tab, the request is taking around 5000-7000 ms. The textItem here supports comma seperated autosuggest.

Please let me know if you need anything else.

I’m using :

browser: Chrome: 15.0.874.106 m (its slow in other browsers also)
SmartGwt version: SC_SNAPSHOT-2011-11-03/EVAL Deployment 2011-11-03

Thanks.

Isomorphic · November 9, 2011, 6:32pm

Right, as we’d guessed, you can see that all the time is being taken up waiting for your database - obtaining connections, executing queries, etc. There is nothing we can optimize here for you - our code is simply waiting for your database.

If you have any doubt of this, set up a local Oracle instance on your machine with the same tables. All of these queries will return instantly. Then you can look into why your database is so slow.

curiousgally · November 10, 2011, 12:20am

Isomorphic,

When i query the same using sql developer its pretty fast i.e close to 1 second only for 600 records.

Please find the attached screen shot

Isomorphic · November 10, 2011, 12:28am

Sorry, we can’t debug the difference between your SQL Developer config, what networks it’s traversing or account it’s using, and your SmartGWT install. But if you look at the logs closely, you’ll see it takes multiple whole seconds just to obtain a connection from Oracle.

Something’s broken, and it’s not SmartGWT. Again, if you want to sanity check this, install Oracle locally so you have a clean environment. Once you’ve verified that SmartGWT with Oracle is normally instant, return to your slower environment and troubleshoot that.

curiousgally · November 10, 2011, 12:32am

Thanks for the detail.

But installing oracle on local wouldn’t be feasible at all. Its a huge database of around 5tb sitting at a far place, so trying to mimic database on local is not possible. But one thing I would like to clarify is, both Sql developer and smartgwt contact the db server over the network only. Also I’m using the same oracle jdbc driver here too.

Thanks;.

Isomorphic · November 10, 2011, 12:35am

You don’t need to replicate the entire 5TB database, just the tables needed for a given query that’s slow.

Another difference is the servlet container - not sure where you’re running SQL Developer.

curiousgally · November 10, 2011, 1:45am

I’m running sql developer on the same machine where I’m doing the development work(smartgwt project).

curiousgally · November 11, 2011, 12:15am

Isomorphic,

If you see it is taking more time to get the connection from the pool initially. Can you shed some light on how the connection pool is managed. Also is there any way to specify these details like number of connections to be allowed, the init SQL etc. Any place where to read about all the properties that could be set in the server.properties.

Thanks.

Isomorphic · November 11, 2011, 12:21am

We mentioned the connection pool delay just to indicate that everything (not just queries) is exceptionally slow with your database. You’re probably better off focusing on why queries are slow.

However if you Google your driver (oracle.jdbc.pool.OracleDataSource) you’ll find Oracle docs on how pooling is configured, and the Admin Console shows you all the properties that can be set for drivers in general (but these do not have to do with pooling).