Learn more about SQL Server tools

mssqltips logo
 

Tutorials          DBA          Dev          BI          Career          Categories          Webcasts          Whitepapers          Today's Tip          Join

Tutorials      DBA      Dev      BI      Categories      Webcasts

DBA    Dev    BI    Categories

Special Announcement: SQL Server Performance Tuning Tips and Tricks Webinar
 

Crawled and Managed properties in MOSS 2007


By:   |   Read Comments   |   Related Tips: > Sharepoint

Problem

I understand that SharePoint lists and libraries contain extra columns (i.e. Author, Description, Status etc) that can be used to group, sort and filter content items as needed. These extra columns are known as "Metadata".

I also understand that metadata is collected by SharePoint's search crawler and can be used in various ways by developers and administrators. I would like to understand more about this.

Solution

Crawled Properties: All the site columns(i.e. metadata) are picked by the Search Crawler based on the protocol handler used.

Notes:

  • Site columns with blank data are not picked by Search Crawler. The site columns must have some content in order to be picked by Search Crawler as Crawled Property. In case, you run into this issue, add some content (at least 1 item) and do an incremental or full crawl.
  • Search Crawls normally take a good amount of time if the amount of content is large.

Crawled properties can be seen in the Shared Services Provider’s Metadata Properties page. Launch SharePoint 3.0 Central Administration -> Select Shared Services Administration -> Select the Shared Services Provider -> In the Shared Services Provider, you can see the categories of the Crawled Properties by one of the following two ways:

  • Search Settings
  • Search Administration (if the Infrastructure Update is installed)
shared services

Click on any of those 2 links. Click on the "Metadata Properties" under the "Queries and Results" heading in the left navigation.

administration

Click on "Crawled properties".

crawled properties

There are 11 crawled property categories that are available out of the box. You can see this to understand about all the categories. However, here we are interested in the SharePoint category.

metadata property mappings

Any crawled property that is prefixed with "ows_" comes from SharePoint. This is how it avoids conflict with crawled properties from other categories.

sharepoint

Content Sources: A content source specifies settings that define how and on what schedule content is crawled. It includes one or more addresses of a content repository from which to start crawling, also called "start addresses". These settings apply to all start addresses within the entire content source. Let us create a content source which can be used to crawl the site collection, which contains the Tips column data.

  • Click on the link "Content Sources" in the left navigation and then click on "New Content Source".
content sources

Create a content source. Give it a name and the start address of your site collection.

portal web app

Click on the arrow next to the new content source you created and start a full crawl.

start full crawl

The crawl will take some time depending on the amount of data you have in your site collection

portal web app

Keep refreshing the page until you see the crawl is complete and the status is set to "Idle".

content source

Now, the crawler would have picked new site columns which has some content. Let us verify. Go to Crawled properties -> Sharepoint category. Search for the property "ows_Tips" (remember the crawler adds "ows_" as prefix to Sharepoint crawled properties.

property type

Metadata Properties: The Metadata property mappings, map properties extracted from documents during crawls, called crawled properties, to managed properties that users can use in search queries. Now in order to use the "Tips" crawled property in a search, we need to create a metadata property and map it to the "Tips" Crawled properties

Click on Metadata properties in left nav. Click on "New Managed Property"

managed properties view

Give a name to Metadata property (in this example - Tips) and click on Add Mapping. Map this to "ows_Tips" Crawled property. Make sure that you check the box "Allow this property to be used in scopes". Doing so, will make this managed property available for search scoping (this will be explained later in this article).

items

Please note that after mapping the property, you need to do another crawl for the content to be picked. As you see in the figure below, the number of items found in this property shows 0.

property name

After the crawl is complete, it will show the actual count as shown below:

tips

Now the metadata property is ready for searching. There are various ways you can use the metadata property for search requirements. A few options include Basic search, Advanced search, search query etc.

Use in Search scope:

Go to Site -> Site Actions -> Site Settings -> Modify All site settings -> Click on Search scopes under Administration.

search scopes

Click on create "new scope".

view scopes

Create a scope (Tips).

tips

Now we will have to create a rule for this scope. Click on Add rules.

view scopes

You can use the property query option to set rules specific to the metadata. For example, let us say we want to search only content which has the Tips column value equal to "Indexing".

add scope rule
property query

Please note that only those properties are shown in the drop down, which have its crawled property set to "Allow this property to be used in scopes" (as explained above).

After you click OK, the scope takes some time to get updated.

view scopes

After the scope is ready, you can see the number to items available under this scope.

view scopes

As you can see below, this search scope is available in the basic search scope dropdown.

sytem account

Now going forward, whenever data is added in the list, only an incremental crawl is required for it to be available for searching. For example, let's add one record in the list as below.

sample

Go to search SSP Admin -> content source and do an incremental crawl.

start incremental crawl

It should be quicker than a full crawl as it just has to pick the changed data from the last crawl instead of crawling the all of the content.

crawling incremental

After the crawl is complete, you can go to the scope and see that the count has been increased to 2.

view scopes

Use in "Advanced Search": Metadata properties can be used to narrow down search results in Advance Search. As you see in the figure below, the Tips scope is available under "Narrow the search" section.

tips

We can also add a custom property restriction in Advanced Search. To do this, we need to modify the advanced search property XML statement.

advanced search box

Please remember to add both the PropertyDef and PropertyRef nodes. It is recommended to edit the XML in Visual Studio or some XML editor.

property name

Use in a custom search query: These metadata properties can be used as columns when writing search queries (either through the API or the search web service). Below is an example for a search query using FullTextSqlQuery

sql query

Planning search is an important part of any Microsoft Office SharePoint Server 2007 deployment. If you devote time to plan how search is implemented, you can save time in the future and can reinforce the effectiveness of other business processes. For targeted and large environments, Metadata and Crawled properties are key pieces to SharePoint search.

Next Steps


Last Update:


next webcast button


next tip button



About the author





Post a comment or let the author know this tip helped.

All comments are reviewed, so stay on subject or we may delete your comment. Note: your email address is not published. Required fields are marked with an asterisk (*).

*Name    *Email    Email me updates 


Signup for our newsletter
 I agree by submitting my data to receive communications, account updates and/or special offers about SQL Server from MSSQLTips and/or its Sponsors. I have read the privacy statement and understand I may unsubscribe at any time.



    



Learn more about SQL Server tools