Connecting the out-of-sample and pre-image problems in kernel methods

Arias, Pablo - Randall, Gregory - Sapiro, Guillermo

Resumen:

Kernel methods have been widely studied in the field of pattern recognition. These methods implicitly map, "the kernel trick," the data into a space which is more appropriate for analysis. Many manifold learning and dimensionality reduction techniques are simply kernel methods for which the mapping is explicitly computed. In such cases, two problems related with the mapping arise: The out-of-sample extension and the pre-image computation. In this paper we propose a new pre-image method based on the Nystrom formulation for the out-of-sample extension, showing the connections between both problems. We also address the importance of normalization in the feature space, which has been ignored by standard pre-image algorithms. As an example, we apply these ideas to the Gaussian kernel, and relate our approach to other popular pre-image methods. Finally, we show the application of these techniques in the study of dynamic shapes.


Detalles Bibliográficos
2007
Feature extraction
Gaussian processes
Image recognition
Inglés
Universidad de la República
COLIBRI
https://hdl.handle.net/20.500.12008/38759
Acceso abierto
Licencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0)
_version_ 1807522935097786368
author Arias, Pablo
author2 Randall, Gregory
Sapiro, Guillermo
author2_role author
author
author_facet Arias, Pablo
Randall, Gregory
Sapiro, Guillermo
author_role author
bitstream.checksum.fl_str_mv 7f2e2c17ef6585de66da58d1bfa8b5e1
9833653f73f7853880c94a6fead477b1
4afdbb8c545fd630ea7db775da747b2f
9da0b6dfac957114c6a7714714b86306
39cfac49304a2cbfafc776e0c557a418
3aefb4b8c0638e2c094fb2d0f5aada3a
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
MD5
MD5
MD5
bitstream.url.fl_str_mv http://localhost:8080/xmlui/bitstream/20.500.12008/38759/6/license.txt
http://localhost:8080/xmlui/bitstream/20.500.12008/38759/3/license_text
http://localhost:8080/xmlui/bitstream/20.500.12008/38759/4/license_url
http://localhost:8080/xmlui/bitstream/20.500.12008/38759/5/license_rdf
http://localhost:8080/xmlui/bitstream/20.500.12008/38759/1/ARS07.pdf
http://localhost:8080/xmlui/bitstream/20.500.12008/38759/2/Correction+to+Arias+et+al.+%5B1%5D.pdf
collection COLIBRI
dc.creator.none.fl_str_mv Arias, Pablo
Randall, Gregory
Sapiro, Guillermo
dc.date.accessioned.none.fl_str_mv 2023-08-01T20:33:39Z
dc.date.available.none.fl_str_mv 2023-08-01T20:33:39Z
dc.date.issued.es.fl_str_mv 2007
dc.date.submitted.es.fl_str_mv 20230801
dc.description.abstract.none.fl_txt_mv Kernel methods have been widely studied in the field of pattern recognition. These methods implicitly map, "the kernel trick," the data into a space which is more appropriate for analysis. Many manifold learning and dimensionality reduction techniques are simply kernel methods for which the mapping is explicitly computed. In such cases, two problems related with the mapping arise: The out-of-sample extension and the pre-image computation. In this paper we propose a new pre-image method based on the Nystrom formulation for the out-of-sample extension, showing the connections between both problems. We also address the importance of normalization in the feature space, which has been ignored by standard pre-image algorithms. As an example, we apply these ideas to the Gaussian kernel, and relate our approach to other popular pre-image methods. Finally, we show the application of these techniques in the study of dynamic shapes.
dc.description.es.fl_txt_mv Trabajo presentado a la IEEE Conference on Computer Vision and Pattern Recognition, 2007
dc.identifier.citation.es.fl_str_mv Arias, P., Randall, G., Sapiro, G. Connecting the out-of-sample and pre-Image problems in Kernel methods [Preprint] Publicado en IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007. doi: 10.1109/CVPR.2007.383038.
dc.identifier.uri.none.fl_str_mv https://hdl.handle.net/20.500.12008/38759
dc.language.iso.none.fl_str_mv en
eng
dc.rights.license.none.fl_str_mv Licencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0)
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
dc.source.none.fl_str_mv reponame:COLIBRI
instname:Universidad de la República
instacron:Universidad de la República
dc.subject.es.fl_str_mv Feature extraction
Gaussian processes
Image recognition
dc.title.none.fl_str_mv Connecting the out-of-sample and pre-image problems in kernel methods
dc.type.es.fl_str_mv Preprint
dc.type.none.fl_str_mv info:eu-repo/semantics/preprint
dc.type.version.none.fl_str_mv info:eu-repo/semantics/submittedVersion
description Trabajo presentado a la IEEE Conference on Computer Vision and Pattern Recognition, 2007
eu_rights_str_mv openAccess
format preprint
id COLIBRI_aa063195ae9548596e0764765d60349a
identifier_str_mv Arias, P., Randall, G., Sapiro, G. Connecting the out-of-sample and pre-Image problems in Kernel methods [Preprint] Publicado en IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007. doi: 10.1109/CVPR.2007.383038.
instacron_str Universidad de la República
institution Universidad de la República
instname_str Universidad de la República
language eng
language_invalid_str_mv en
network_acronym_str COLIBRI
network_name_str COLIBRI
oai_identifier_str oai:colibri.udelar.edu.uy:20.500.12008/38759
publishDate 2007
reponame_str COLIBRI
repository.mail.fl_str_mv mabel.seroubian@seciu.edu.uy
repository.name.fl_str_mv COLIBRI - Universidad de la República
repository_id_str 4771
rights_invalid_str_mv Licencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0)
spelling 2023-08-01T20:33:39Z2023-08-01T20:33:39Z200720230801Arias, P., Randall, G., Sapiro, G. Connecting the out-of-sample and pre-Image problems in Kernel methods [Preprint] Publicado en IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007. doi: 10.1109/CVPR.2007.383038.https://hdl.handle.net/20.500.12008/38759Trabajo presentado a la IEEE Conference on Computer Vision and Pattern Recognition, 2007Kernel methods have been widely studied in the field of pattern recognition. These methods implicitly map, "the kernel trick," the data into a space which is more appropriate for analysis. Many manifold learning and dimensionality reduction techniques are simply kernel methods for which the mapping is explicitly computed. In such cases, two problems related with the mapping arise: The out-of-sample extension and the pre-image computation. In this paper we propose a new pre-image method based on the Nystrom formulation for the out-of-sample extension, showing the connections between both problems. We also address the importance of normalization in the feature space, which has been ignored by standard pre-image algorithms. As an example, we apply these ideas to the Gaussian kernel, and relate our approach to other popular pre-image methods. Finally, we show the application of these techniques in the study of dynamic shapes.Made available in DSpace on 2023-08-01T20:33:39Z (GMT). No. of bitstreams: 6 ARS07.pdf: 214968 bytes, checksum: 39cfac49304a2cbfafc776e0c557a418 (MD5) Correction to Arias et al. [1].pdf: 12972 bytes, checksum: 3aefb4b8c0638e2c094fb2d0f5aada3a (MD5) license_text: 21936 bytes, checksum: 9833653f73f7853880c94a6fead477b1 (MD5) license_url: 49 bytes, checksum: 4afdbb8c545fd630ea7db775da747b2f (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) license.txt: 4194 bytes, checksum: 7f2e2c17ef6585de66da58d1bfa8b5e1 (MD5) Previous issue date: 2007enengLas obras depositadas en el Repositorio se rigen por la Ordenanza de los Derechos de la Propiedad Intelectual de la Universidad De La República. (Res. Nº 91 de C.D.C. de 8/III/1994 – D.O. 7/IV/1994) y por la Ordenanza del Repositorio Abierto de la Universidad de la República (Res. Nº 16 de C.D.C. de 07/10/2014)info:eu-repo/semantics/openAccessLicencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0)Feature extractionGaussian processesImage recognitionConnecting the out-of-sample and pre-image problems in kernel methodsPreprintinfo:eu-repo/semantics/preprintinfo:eu-repo/semantics/submittedVersionreponame:COLIBRIinstname:Universidad de la Repúblicainstacron:Universidad de la RepúblicaArias, PabloRandall, GregorySapiro, GuillermoLICENSElicense.txttext/plain4194http://localhost:8080/xmlui/bitstream/20.500.12008/38759/6/license.txt7f2e2c17ef6585de66da58d1bfa8b5e1MD56CC-LICENSElicense_textapplication/octet-stream21936http://localhost:8080/xmlui/bitstream/20.500.12008/38759/3/license_text9833653f73f7853880c94a6fead477b1MD53license_urlapplication/octet-stream49http://localhost:8080/xmlui/bitstream/20.500.12008/38759/4/license_url4afdbb8c545fd630ea7db775da747b2fMD54license_rdfapplication/octet-stream23148http://localhost:8080/xmlui/bitstream/20.500.12008/38759/5/license_rdf9da0b6dfac957114c6a7714714b86306MD55ORIGINALARS07.pdfapplication/pdf214968http://localhost:8080/xmlui/bitstream/20.500.12008/38759/1/ARS07.pdf39cfac49304a2cbfafc776e0c557a418MD51Correction to Arias et al. [1].pdfapplication/pdf12972http://localhost:8080/xmlui/bitstream/20.500.12008/38759/2/Correction+to+Arias+et+al.+%5B1%5D.pdf3aefb4b8c0638e2c094fb2d0f5aada3aMD5220.500.12008/387592023-08-01 17:33:39.315oai:colibri.udelar.edu.uy:20.500.12008/38759VGVybWlub3MgeSBjb25kaWNpb25lcyByZWxhdGl2YXMgYWwgZGVwb3NpdG8gZGUgb2JyYXMKCgpMYXMgb2JyYXMgZGVwb3NpdGFkYXMgZW4gZWwgUmVwb3NpdG9yaW8gc2UgcmlnZW4gcG9yIGxhIE9yZGVuYW56YSBkZSBsb3MgRGVyZWNob3MgZGUgbGEgUHJvcGllZGFkIEludGVsZWN0dWFsICBkZSBsYSBVbml2ZXJzaWRhZCBEZSBMYSBSZXDvv71ibGljYS4gKFJlcy4gTu+/vSA5MSBkZSBDLkQuQy4gZGUgOC9JSUkvMTk5NCDvv70gRC5PLiA3L0lWLzE5OTQpIHkgIHBvciBsYSBPcmRlbmFuemEgZGVsIFJlcG9zaXRvcmlvIEFiaWVydG8gZGUgbGEgVW5pdmVyc2lkYWQgZGUgbGEgUmVw77+9YmxpY2EgKFJlcy4gTu+/vSAxNiBkZSBDLkQuQy4gZGUgMDcvMTAvMjAxNCkuIAoKQWNlcHRhbmRvIGVsIGF1dG9yIGVzdG9zIHTvv71ybWlub3MgeSBjb25kaWNpb25lcyBkZSBkZXDvv71zaXRvIGVuIENPTElCUkksIGxhIFVuaXZlcnNpZGFkIGRlIFJlcO+/vWJsaWNhIHByb2NlZGVy77+9IGE6ICAKCmEpIGFyY2hpdmFyIG3vv71zIGRlIHVuYSBjb3BpYSBkZSBsYSBvYnJhIGVuIGxvcyBzZXJ2aWRvcmVzIGRlIGxhIFVuaXZlcnNpZGFkIGEgbG9zIGVmZWN0b3MgZGUgZ2FyYW50aXphciBhY2Nlc28sIHNlZ3VyaWRhZCB5IHByZXNlcnZhY2nvv71uCmIpIGNvbnZlcnRpciBsYSBvYnJhIGEgb3Ryb3MgZm9ybWF0b3Mgc2kgZnVlcmEgbmVjZXNhcmlvICBwYXJhIGZhY2lsaXRhciBzdSBwcmVzZXJ2YWNp77+9biB5IGFjY2VzaWJpbGlkYWQgc2luIGFsdGVyYXIgc3UgY29udGVuaWRvLgpjKSByZWFsaXphciBsYSBjb211bmljYWNp77+9biBw77+9YmxpY2EgeSBkaXNwb25lciBlbCBhY2Nlc28gbGlicmUgeSBncmF0dWl0byBhIHRyYXbvv71zIGRlIEludGVybmV0IG1lZGlhbnRlIGxhIHB1YmxpY2Fjae+/vW4gZGUgbGEgb2JyYSBiYWpvIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgc2VsZWNjaW9uYWRhIHBvciBlbCBwcm9waW8gYXV0b3IuCgoKRW4gY2FzbyBxdWUgZWwgYXV0b3IgaGF5YSBkaWZ1bmRpZG8geSBkYWRvIGEgcHVibGljaWRhZCBhIGxhIG9icmEgZW4gZm9ybWEgcHJldmlhLCAgcG9kcu+/vSBzb2xpY2l0YXIgdW4gcGVy77+9b2RvIGRlIGVtYmFyZ28gc29icmUgbGEgZGlzcG9uaWJpbGlkYWQgcO+/vWJsaWNhIGRlIGxhIG1pc21hLCBlbCBjdWFsIGNvbWVuemFy77+9IGEgcGFydGlyIGRlIGxhIGFjZXB0YWNp77+9biBkZSBlc3RlIGRvY3VtZW50byB5IGhhc3RhIGxhIGZlY2hhIHF1ZSBpbmRpcXVlIC4KCkVsIGF1dG9yIGFzZWd1cmEgcXVlIGxhIG9icmEgbm8gaW5mcmlnZSBuaW5n77+9biBkZXJlY2hvIHNvYnJlIHRlcmNlcm9zLCB5YSBzZWEgZGUgcHJvcGllZGFkIGludGVsZWN0dWFsIG8gY3VhbHF1aWVyIG90cm8uCgpFbCBhdXRvciBnYXJhbnRpemEgcXVlIHNpIGVsIGRvY3VtZW50byBjb250aWVuZSBtYXRlcmlhbGVzIGRlIGxvcyBjdWFsZXMgbm8gdGllbmUgbG9zIGRlcmVjaG9zIGRlIGF1dG9yLCAgaGEgb2J0ZW5pZG8gZWwgcGVybWlzbyBkZWwgcHJvcGlldGFyaW8gZGUgbG9zIGRlcmVjaG9zIGRlIGF1dG9yLCB5IHF1ZSBlc2UgbWF0ZXJpYWwgY3V5b3MgZGVyZWNob3Mgc29uIGRlIHRlcmNlcm9zIGVzdO+/vSBjbGFyYW1lbnRlIGlkZW50aWZpY2FkbyB5IHJlY29ub2NpZG8gZW4gZWwgdGV4dG8gbyBjb250ZW5pZG8gZGVsIGRvY3VtZW50byBkZXBvc2l0YWRvIGVuIGVsIFJlcG9zaXRvcmlvLgoKRW4gb2JyYXMgZGUgYXV0b3Lvv71hIG3vv71sdGlwbGUgL3NlIHByZXN1bWUvIHF1ZSBlbCBhdXRvciBkZXBvc2l0YW50ZSBkZWNsYXJhIHF1ZSBoYSByZWNhYmFkbyBlbCBjb25zZW50aW1pZW50byBkZSB0b2RvcyBsb3MgYXV0b3JlcyBwYXJhIHB1YmxpY2FybGEgZW4gZWwgUmVwb3NpdG9yaW8sIHNpZW5kbyDvv71zdGUgZWwg77+9bmljbyByZXNwb25zYWJsZSBmcmVudGUgYSBjdWFscXVpZXIgdGlwbyBkZSByZWNsYW1hY2nvv71uIGRlIGxvcyBvdHJvcyBjb2F1dG9yZXMuCgpFbCBhdXRvciBzZXLvv70gcmVzcG9uc2FibGUgZGVsIGNvbnRlbmlkbyBkZSBsb3MgZG9jdW1lbnRvcyBxdWUgZGVwb3NpdGEuIExhIFVERUxBUiBubyBzZXLvv70gcmVzcG9uc2FibGUgcG9yIGxhcyBldmVudHVhbGVzIHZpb2xhY2lvbmVzIGFsIGRlcmVjaG8gZGUgcHJvcGllZGFkIGludGVsZWN0dWFsIGVuIHF1ZSBwdWVkYSBpbmN1cnJpciBlbCBhdXRvci4KCkFudGUgY3VhbHF1aWVyIGRlbnVuY2lhIGRlIHZpb2xhY2nvv71uIGRlIGRlcmVjaG9zIGRlIHByb3BpZWRhZCBpbnRlbGVjdHVhbCwgbGEgVURFTEFSICBhZG9wdGFy77+9IHRvZGFzIGxhcyBtZWRpZGFzIG5lY2VzYXJpYXMgcGFyYSBldml0YXIgbGEgY29udGludWFjae+/vW4gZGUgZGljaGEgaW5mcmFjY2nvv71uLCBsYXMgcXVlIHBvZHLvv71uIGluY2x1aXIgZWwgcmV0aXJvIGRlbCBhY2Nlc28gYSBsb3MgY29udGVuaWRvcyB5L28gbWV0YWRhdG9zIGRlbCBkb2N1bWVudG8gcmVzcGVjdGl2by4KCkxhIG9icmEgc2UgcG9uZHLvv70gYSBkaXNwb3NpY2nvv71uIGRlbCBw77+9YmxpY28gYSB0cmF277+9cyBkZSBsYXMgbGljZW5jaWFzIENyZWF0aXZlIENvbW1vbnMsIGVsIGF1dG9yIHBvZHLvv70gc2VsZWNjaW9uYXIgdW5hIGRlIGxhcyA2IGxpY2VuY2lhcyBkaXNwb25pYmxlczoKCgpBdHJpYnVjae+/vW4gKENDIC0gQnkpOiBQZXJtaXRlIHVzYXIgbGEgb2JyYSB5IGdlbmVyYXIgb2JyYXMgZGVyaXZhZGFzLCBpbmNsdXNvIGNvbiBmaW5lcyBjb21lcmNpYWxlcywgc2llbXByZSBxdWUgc2UgcmVjb25vemNhIGFsIGF1dG9yLgoKQXRyaWJ1Y2nvv71uIO+/vSBDb21wYXJ0aXIgSWd1YWwgKENDIC0gQnktU0EpOiBQZXJtaXRlIHVzYXIgbGEgb2JyYSB5IGdlbmVyYXIgb2JyYXMgZGVyaXZhZGFzLCBpbmNsdXNvIGNvbiBmaW5lcyBjb21lcmNpYWxlcywgcGVybyBsYSBkaXN0cmlidWNp77+9biBkZSBsYXMgb2JyYXMgZGVyaXZhZGFzIGRlYmUgaGFjZXJzZSBtZWRpYW50ZSB1bmEgbGljZW5jaWEgaWTvv71udGljYSBhIGxhIGRlIGxhIG9icmEgb3JpZ2luYWwsIHJlY29ub2NpZW5kbyBhIGxvcyBhdXRvcmVzLgoKQXRyaWJ1Y2nvv71uIO+/vSBObyBDb21lcmNpYWwgKENDIC0gQnktTkMpOiBQZXJtaXRlIHVzYXIgbGEgb2JyYSB5IGdlbmVyYXIgb2JyYXMgZGVyaXZhZGFzLCBzaWVtcHJlIHkgY3VhbmRvIGVzb3MgdXNvcyBubyB0ZW5nYW4gZmluZXMgY29tZXJjaWFsZXMsIHJlY29ub2NpZW5kbyBhbCBhdXRvci4KCkF0cmlidWNp77+9biDvv70gU2luIERlcml2YWRhcyAoQ0MgLSBCeS1ORCk6IFBlcm1pdGUgZWwgdXNvIGRlIGxhIG9icmEsIGluY2x1c28gY29uIGZpbmVzIGNvbWVyY2lhbGVzLCBwZXJvIG5vIHNlIHBlcm1pdGUgZ2VuZXJhciBvYnJhcyBkZXJpdmFkYXMsIGRlYmllbmRvIHJlY29ub2NlciBhbCBhdXRvci4KCkF0cmlidWNp77+9biDvv70gTm8gQ29tZXJjaWFsIO+/vSBDb21wYXJ0aXIgSWd1YWwgKENDIO+/vSBCeS1OQy1TQSk6IFBlcm1pdGUgdXNhciBsYSBvYnJhIHkgZ2VuZXJhciBvYnJhcyBkZXJpdmFkYXMsIHNpZW1wcmUgeSBjdWFuZG8gZXNvcyB1c29zIG5vIHRlbmdhbiBmaW5lcyBjb21lcmNpYWxlcyB5IGxhIGRpc3RyaWJ1Y2nvv71uIGRlIGxhcyBvYnJhcyBkZXJpdmFkYXMgc2UgaGFnYSBtZWRpYW50ZSBsaWNlbmNpYSBpZO+/vW50aWNhIGEgbGEgZGUgbGEgb2JyYSBvcmlnaW5hbCwgcmVjb25vY2llbmRvIGEgbG9zIGF1dG9yZXMuCgpBdHJpYnVjae+/vW4g77+9IE5vIENvbWVyY2lhbCDvv70gU2luIERlcml2YWRhcyAoQ0MgLSBCeS1OQy1ORCk6IFBlcm1pdGUgdXNhciBsYSBvYnJhLCBwZXJvIG5vIHNlIHBlcm1pdGUgZ2VuZXJhciBvYnJhcyBkZXJpdmFkYXMgeSBubyBzZSBwZXJtaXRlIHVzbyBjb24gZmluZXMgY29tZXJjaWFsZXMsIGRlYmllbmRvIHJlY29ub2NlciBhbCBhdXRvci4KCkxvcyB1c29zIHByZXZpc3RvcyBlbiBsYXMgbGljZW5jaWFzIGluY2x1eWVuIGxhIGVuYWplbmFjae+/vW4sIHJlcHJvZHVjY2nvv71uLCBjb211bmljYWNp77+9biwgcHVibGljYWNp77+9biwgZGlzdHJpYnVjae+/vW4geSBwdWVzdGEgYSBkaXNwb3NpY2nvv71uIGRlbCBw77+9YmxpY28uIExhIGNyZWFjae+/vW4gZGUgb2JyYXMgZGVyaXZhZGFzIGluY2x1eWUgbGEgYWRhcHRhY2nvv71uLCB0cmFkdWNjae+/vW4geSBlbCByZW1peC4KCkN1YW5kbyBzZSBzZWxlY2Npb25lIHVuYSBsaWNlbmNpYSBxdWUgaGFiaWxpdGUgdXNvcyBjb21lcmNpYWxlcywgZWwgZGVw77+9c2l0byBkZWJlcu+/vSBzZXIgYWNvbXBh77+9YWRvIGRlbCBhdmFsIGRlbCBqZXJhcmNhIG3vv714aW1vIGRlbCBTZXJ2aWNpbyBjb3JyZXNwb25kaWVudGUuCgoKCgoKCgoKUniversidadhttps://udelar.edu.uy/https://www.colibri.udelar.edu.uy/oai/requestmabel.seroubian@seciu.edu.uyUruguayopendoar:47712024-07-25T14:33:29.769070COLIBRI - Universidad de la Repúblicafalse
spellingShingle Connecting the out-of-sample and pre-image problems in kernel methods
Arias, Pablo
Feature extraction
Gaussian processes
Image recognition
status_str submittedVersion
title Connecting the out-of-sample and pre-image problems in kernel methods
title_full Connecting the out-of-sample and pre-image problems in kernel methods
title_fullStr Connecting the out-of-sample and pre-image problems in kernel methods
title_full_unstemmed Connecting the out-of-sample and pre-image problems in kernel methods
title_short Connecting the out-of-sample and pre-image problems in kernel methods
title_sort Connecting the out-of-sample and pre-image problems in kernel methods
topic Feature extraction
Gaussian processes
Image recognition
url https://hdl.handle.net/20.500.12008/38759