Monday, November 24, 2014

Extract wsp packages which are currently installed in the Sharepoint solution store via PowerShell

During installation of updates to the Sharepoint environment, especially on production environment, it is good to have rollback plan. One of the step of this rollback plan is to restore wsp packages which were used before installation. Here is the PowerShell script which will enumerate and save all wsps packages (farm solutions) currently installed to the solutions store to the local file system:

   1: (Get-SPFarm).Solutions | ForEach-Object { $var = (Get-Location).Path +
   2:     "\" + $_.Name; $_.SolutionFile.SaveAs($var) }

(you need to use it as one line script. I added line break for better readability). Having this script you may backup farm solutions wsps before to perform update and rollback them if needed.

Saturday, November 22, 2014

Extend Sharepoint 2013 search queries with custom query variables populated dynamically in runtime

Sharepoint 2013 allows to extend search queries via several ways. The commonly used way is to do it via Search result sources:


In the query builder we may add more conditions to those which user enters in the search box. They are applied to query when user performs search and ResultScriptWebPart or ContentBySearchWebPart are configured to use appropriate search result source. It can be done via web part properties for specific web part or for all web parts if search result source is set as default. If you need to create custom result source and set it default programmatically you may check the following article: Create custom search result source programmatically in Sharepoint 2013. Note that there is one problem related with custom default result sources which I wrote about here: Problem with missing catalog connections when use customized search result source in Sharepoint 2013.

This is useful approach when we need to add more conditions to the search query. Using search query variables we may retrieve values from different site or site collection properties (e.g. id or url), current user’s properties, managed metadata term used for current navigation node and others. Full list of available standard query variables can be found here: Query variables in SharePoint Server 2013. However anyway with standard query variables the set of values which we may use in the queries is quite limited. But what if we need to add condition to the query with value which should be calculated dynamically in runtime, e.g. if we need to add value from cookie or value calculated by complex business logic?

One of the way is to use query string variable {QueryString.<ParameterName>}, where <ParameterName> should be replaced with the actual query string parameter name ,e.g. {QueryString.Country}. On practice however it is not always convenient or possible. E.g. by default search result web part (ResultScriptWebPart) uses client navigation (can be controlled via UpdateAjaxNavigate property), i.e. adds search phrase entered by user in search box on the search results page to the current url after hash “#” symbol. E.g. if user enters something in the search box on site’s front page and clicks search, user is redirected to the search result page with search phrase specified in “k” query string parameter:

But if after that user adds “bar” phrase to the search box on the search results page, url will look like this:

and ResultScriptWebPart will use the last phrase defined after hash symbol. The problem however that part after hash tag is not considered by the standard as part of url and thus is not sent to the server (see the following forum thread for more details: How to get Url Hash (#) from server side). For our example it means that query string variable sent to the server will be empty (note that there is also one problem with empty query string variables described here: One problem with “Value of a parameter from URL” query variable in Sharepoint 2013 search queries) and search won’t work as we expect.

In order to add dynamically calculated values to the search query we may also try to use http request properties {Request.<PropertyName>}, but as people mentioned in the forums, it really works only with Url property (although I didn’t check it by myself), so you can use e.g. request’s property bag for that.

So what to do when we need to add dynamic value to the search query? In this case the following approach can be used:

1. add necessary condition to the search query builder with custom variable name:


Here we added condition that managed property MyManagedProperty should be equal to {Country} variable. {Country} – is custom variable which will be added to the query processing pipeline by custom code (see below).

2. implement custom web part and put it on the page with ResultScriptWebPart or ContentBySearchWebPart and implement the following code there:

   1: // retrieve value from cookies or from any other place
   2: string country = ...;
   4: var scriptApp = ScriptApplicationManager.GetCurrent(this.Page);
   5: var dataProvider = scriptApp.QueryGroups["Default"].DataProvider;
   6: dataProvider.BeforeSerializeToClient += (sender, args) =>
   7:     {
   8:         var dp = sender as DataProviderScriptWebPart;
   9:         if (dp == null)
  10:         {
  11:             return;
  12:         }
  14:         dp.Properties["Country"] = country;
  15:     };

After that you search query will return only those content which has MyManagedProperty set to particular country, which is retrieved from the cookies or calculated using any other business logic. All credits should go to Mikael Svenson from which article I’ve got the idea of using this approach. Hope that this information will help you in your work.

Thursday, November 13, 2014

Problem with updating managed properties in display templates used in Sharepoint search

Display templates are files with .html extension which are used for customizing appearance of the content in Sharepoint search results. When you upload such files to Site settings > Master page and page layouts > Display templates folder, Sharepoint automatically creates js version of display template, which is actually used by search engine. When you publish or delete basic html file, associated js file is published or deleted automatically.

The structure of most display templates looks like this:

   1: <html xmlns:mso="urn:schemas-microsoft-com:office:office"
   2: xmlns:msdt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"> 
   3: <head>
   4: <title>Foo</title>
   6: <!--[if gte mso 9]><xml>
   7: <mso:CustomDocumentProperties>
   8: <mso:TemplateHidden msdt:dt="string">0</mso:TemplateHidden>
   9: <mso:MasterPageDescription msdt:dt="string">Foo</mso:MasterPageDescription>
  10: <mso:ContentTypeId msdt:dt="string">...</mso:ContentTypeId>
  11: <mso:TargetControlType msdt:dt="string">;#SearchResults;#</mso:TargetControlType>
  12: <mso:HtmlDesignAssociated msdt:dt="string">1</mso:HtmlDesignAssociated>
  13: <mso:ManagedPropertyMapping msdt:dt="string">'Title':'Title','Path':'Path',
  14: 'Description':'Description','EditorOWSUSER':'EditorOWSUSER',
  15: 'LastModifiedTime':'LastModifiedTime','CollapsingStatus':'CollapsingStatus',
  16: 'DocId':'DocId','HitHighlightedSummary':'HitHighlightedSummary',
  17: 'HitHighlightedProperties':'HitHighlightedProperties',
  18: 'FileExtension':'FileExtension','ViewsLifeTime':'ViewsLifeTime',
  19: 'ParentLink':'ParentLink','FileType':'FileType','IsContainer':'IsContainer',
  20: 'SecondaryFileExtension':'SecondaryFileExtension','DisplayAuthor':'DisplayAuthor',
  21: 'SPSiteURL':'SPSiteURL',...</mso:ManagedPropertyMapping>
  22: </mso:CustomDocumentProperties>
  23: </xml><![endif]-->
  24: </head>
  25: <body>
  26:     <div id="Item_Foo">
  27:         ...
  28:     </div>
  29: </body>
  30: </html>

In the top area of the file metadata section is defined. Very important section is <mso:ManagedPropertyMapping>…</<mso:ManagedPropertyMapping>. It contains list of managed properties which can be used in the current display template. The actual look and feel template is defined inside topmost div tag (in example above it’s the div with id = “Item_Foo”).

During the lifetime of your application it may be needed to update existing display templates which are used in living site. If you only need to update look and feel part of template it won’t cause a lot of problems: just re-upload html file to the the location inside Site settings > Master page and page layouts > Display templates folder and publish it (depending on your settings you may also need to approve the file). But if you added new managed properties to <mso:ManagedPropertyMapping>…</<mso:ManagedPropertyMapping> section you may face with the problem that values of these properties will be always empty when you will try to use them inside display template.

This problem is caused by Sharepoint cache: it somehow caches mapped managed properties so deeply, that simple re-uploading and re-publishing doesn’t help (I tried also restart Sharepoint search and search host services and restart the server, but it didn’t help). In order to clear them from cache I found the following solution: delete original html file from Site settings > Master page and page layouts > Display templates, upload it from scratch and publish. If this display template was used in some search result types, you may also need to re-create this result type. After that try to search the content, for which updated display template should be used (Ctrl-F5 may be needed in browser as well): new managed properties should appear now.

Tuesday, November 11, 2014

PowerShell script for enumerating all web parts in Sharepoint web application

Sometimes for maintenance we need to enumerate all web parts in Sharepoint web application. The following script can be used for that:

   1: param( 
   2:     [string]$webAppUrl
   3: )
   5: function Enum-Web-Parts-On-Page($item)
   6: {
   7:     $file = $item.File
   8:     Write-Host "    Enum web parts on page" $file.Url -foregroundcolor green
  10:     $webPartManager = $file.Web.GetLimitedWebPartManager($file.Url,
  11: [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)
  12:     $contains = $false
  13:     foreach ($wp in $webPartManager.WebParts)
  14:     {
  15:         Write-Host "      Web part:" $wp.GetType().FullName  -foregroundcolor green
  16:     }
  17: }
  19: function Enum-Web-Parts-In-Web($w)
  20: {
  21:     Write-Host "  Check pages on web" $w.Url -foregroundcolor green
  23:     $pw = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($w)
  24:     $pagesList = $pw.PagesList
  25:     $pagesList.Items | ForEach-Object { Enum-Web-Parts-On-Page $_ }
  26: }
  28: function Enum-Web-Parts-In-SiteCol($site)
  29: {
  30:     Write-Host "Check sub sites in site col" $site.Url -foregroundcolor green
  31:     $site.AllWebs | ForEach-Object { Enum-Web-Parts-In-Web $_ }
  32: }
  34: if (-not $webAppUrl)
  35: {
  36:     Write-Host "Specify web app url in webAppUrl parameter" -foregroundcolor red
  37:     return
  38: }
  40: $wa = Get-SPWebApplication $webAppUrl
  41: $wa.Sites | ForEach-Object { Enum-Web-Parts-In-SiteCol $_ }

It enumerates all site collections in web application (line 41), then all sub sites in each site collection (line 31), all pages in each sub site (line 25) and finally all web parts on each page (lines 10-16). This script can be used e.g. if you need to find web part of particular type or title on all sub sites in your web application and make changes in them. Hope that this script will be useful in your work.

Saturday, November 8, 2014

One problem with “Value of a parameter from URL” query variable in Sharepoint 2013 search queries

There are several ways to customize search queries in Sharepoint. One of them is to edit query in appropriate search result source used in the site collection (Site settings > Search result source). When you will click Launch query builder button popup window will be opened which will allow to modify search query by applying additional filters. You may add keywords filters and property filters there. Property filter means that you will add condition for managed property used in the crawled content. You may use hardcoded value in rvalue for the condition or one of the available query variables which will be replaced with the real value in runtime:


One of the interesting values is “Value of a parameter from URL” as it allows to modify search queries in runtime. Another value which looks applicable for this is “Value of a field on the page”. Based on it’s name you may think that it allows to use html field (e.g. hidden field) on the page, but unfortunately it means metadata field from the current search results page, like specified in Query variables in SharePoint Server 2013:

The value of a field on the page from where the query was issued. For example, if the page from where the query was issued contained a site column named "ContentOwner," specifying {Page.ContentOwner} would allow you to query for the value of "ContentOwner."

So if we need to modify search query in runtime there are not so many options. We may use mentioned “Value of a parameter from URL” or “Value of a token from URL”. But since programmatically it is easier to add query string parameter comparing with constructing URL routes for using tokens, first option is used more frequently.

Condition which uses “Value of a parameter from URL” variable looks like this:


where ManagedProperty should be the name of managed property for which you add the condition and ParamName – name of the query string parameter. Also in this example we used equals (=) operator, while you may also use other available operators (see Keyword Query Language (KQL) syntax reference for examples) e.g.


If in this case we will pass ?title=foo in query string, search condition will be evaluated to Title=foo. This technique looks good, but there are several problems which you need to aware when will use it. First of all query string parameter should be defined before search query parameter (k=…). I.e. if you have search results page and by default search queries looks like this:, then you need to add additional query string parameter before “k” parameter:

Otherwise search will look for the content which contains “bar&title=foo” phrase which is incorrect of course. Note also that when user changes search query on search results page and clicks search, it happens without postback by ajax call which adds new “k” parameter after “#” sign to existing  query string. In our example it will look like this:

In this case {QueryString.Title} will be treat as empty and search won’t result any results. In order to avoid this problem we will need to add our query string parameter after “#” sign as well (also before “k”):

You may do it by overriding javascript function which is called when user clicks search button on the search results page. However this is not the only problem with “Value of a parameter from URL” variable. Another problem is that when you use it in the queries, it is mandatory that it will have at least some value. I.e. simple search query with only “k” parameter won’t return any results. If you will check logs you will find that query parser fails in this case:

Operator Name=ParserExecutor, Message=The processing of item fails with error Exception occurred due to some other exception

and after that:

ExecuteFlowInternal Flow:Microsoft.SharePointSearchProviderFlow Exception: Microsoft.Office.Server.Search.Query.QueryEmptyException: No results are available. Either no query is specified, or the query came from advanced search (Federated Webparts do not support Advanced Search queries).

As it was mentioned workaround for this problem is to add title with at least some value to the query string:

and then in query builder add condition which won’t filter content by value specified in query string in this case, e.g. like this:

{QueryString.Title}=any OR Title={QueryString.Title}

That’s why also it is important to add query string parameter after “#” sign when user clicks search button on search results page. Hope that this information will help someone.

Update 2014-11-19. Further investigation showed that this problem only appears when you use top-level curly braces in your search result source’s query, e.g.:

{?{searchTerms} -ContentClass=urn:content-class:SPSPeople}

But if you remove them

?{searchTerms} -ContentClass=urn:content-class:SPSPeople

then search become working even when query string variable is not specified, while other conditions still work as expected. This still may be a problem because OTB “Local SharePoint Results” result source uses query with top-level braces.