Semantic extraction from images is of growing interest in the field of content based retrieval. Classical algorithms are mainly limited to the use of features such as colour or texture estimated globally over the whole image, or the use of shape description for clearly extractable object boundaries. There is little provision for the searching of images based on any semantic notion, other than by the use of matching keywords which are manually associated with the images. We believe that it is the combination of a number of image properties, estimated locally for individual regions rather than globally, that will create the ability to recognise objects, and from there we can work towards bridging the semantic gap. Using recent advances in scale-space decomposition techniques, it is possible to transform images into trees in scale space, which represent the topology of regions within the image. Using a sieve mechanism we can decompose an image into a discrete scale-space based on the pixel area of regions within the image. Extrema within the image are merged to create new extrema. The merge operations are represented by branches in a tree. The tree then represents the topology of extrema in an image. We propose that complex objects can be found in the trees using a combination of subgraph isomorphism testing and conventional feature matching and hence provide a vehicle for achieving content-based retrieval and navigation for complex objects in images. Explanations are given for the subgraph isomorphism techniques, and how they can be supplemented with feature matching to produce scores for feature-topology matches of images. Finally, we will explain how the scale-trees could be used in combination with a semantic layer to achieve semantic recognition of scenes, by recognition of parts of the topology of a scale-tree as particular objects.