Saturday, July 12, 2014

Asp.Net Ending Response options, Response.End() vs CompleteRequest()

this blog is for any1 who get the ThreadAbortException when caling Response.End(), and then tried the other options such as HttpContext.Current.ApplicationInstance.CompleteRequest() and found out that there was no replacement.

i'll start with the bottom line: if you want to stop right now your response and send it as is you dont have any other option other than Response.End(), period and it's the right thing to throw that EX in that case.

the answer is its because there is the "HTTP pipeline" and the "Asp.Net pipeline", and CompleteRequest() cuts the HTTP pipeline but not the Asp.Net one so everything that is from handlers forwars (including all page events) continues but your part, the asp one, stays in your hands so that no information will be lost, so ur suppose to manage your code currectly, since information lost is really a bad thing, although in most cases we use Response.End() we know what we are losing and we want it.

see this short article about the why not:

so MSDN advice is to implemet flags like this

from MSDN "Causes ASP.NET to bypass all events and filtering in the HTTP pipeline chain of execution and directly execute the EndRequest event."
the best image i could find is this one where all the part in the "Http Modules" is the http pipeline and from there there is everything we know.

now lets start with my little case story, which is relevant since i created a handler dedicated to ajax based forms, and i saw that in all my T/C i have similar logic so i decided to create a smart enough exception class of my own that will logically do the following
1. Send to the client a "client error message"
2. Log everything
3. End the response, not allowing the code to continue

in my handler every T/C used only my custom exception to forward the EX, and just in case i added a T/C on the entire ProcessRequest method, and in the catch i threw again my custom EX.

so the 1st thing that happened was "System.ComponentModel.InvalidEnumArgumentException: The value of argument 'type' (0) is invalid for Enum type 'EventLogEntryType'.Parameter name: type"

this happened (i am writing this because there was no google for this EX) because of my poor implementation here:

public MyException(string message, string clientMessage, Exception ex)
: base(message, ex)
    ClientMessage = clientMessage;
EntryType = EventLogEntryType.Error;

void WriteErrorToLogAndFlush()
  SPSecurity.RunWithElevatedPrivileges(() =>{   
    string sSource = "MyException";
    string sEvent = "MyException: " + this.ToString();
    EventLog.WriteEntry(sSource, sEvent, EntryType);

    if (HttpContext.Current != null) {
        HttpContext.Current.Response.ContentType = "text/plain";
        HttpContext.Current.Response.StatusCode = 500;

   else {
    EventLog.WriteEntry(sSource, "MyException.WriteErrorToLogAndFlush: HttpContext.Current is null", EventLogEntryType.Error);

so as you can see, I called EntryType Before it was declared.

next was the infamous ThreadAbortException, caused by Response.End()

honestly, even after almost 3 years of .NETting and dozens of Response.End()s (and Response.Redirect()) i never noticed that it throws that EX, so in the beginning i was pretty worried, and everywhere (including MSDN and STACKOVERFLOW) i was promted by everybody to use HttpContext.Current.ApplicationInstance.CompleteRequest(), which we will call form now just CompleteRequest(), so i switched and nothing happed, its like i never did it.

lets remeber my scenario in order to understand, what i wanted to achieve is to create a handler and in any case of throwing an exception I wanted the code to stop and return a different response with a basic message to the client while logging the error.
so to mimic that here is a basic code:

public class h1 : IHttpHandler
   public bool IsReusable { get { return false; } }
   public string outputer { get; set; }
   public void ProcessRequest(HttpContext context)
     //this is just to see the difference between server and client flow in the evenviewer
      outputer = "LogEntry ";
      try {
     //here was all my code and methods calls
          try {        
                 context.Response.Write(" JSON ");
              throw new Exception("an OOTB EX or a custom of my own");
          catch (Exception ex) {
              throw new EX2(" HI ", "info", ex);
              HttpContext.Current.Response.Write(" EVIL");
              outputer += "EVIL ";
      catch (EX2 ex) {
         string got = " OK ";
          outputer += "OK ";
      catch (Exception ex) {
          string got = " BAD ";
          outputer += "BAD ";
      HttpContext.Current.Response.Write(" TREVIL");
      outputer += "TREVIL ";
      EventLog.WriteEntry("response ending testings", outputer, EventLogEntryType.Information);
class EX2 : Exception
   public EX2(string clientMessage, string message, Exception ex) : base(message, ex)
   void WriteErrorToLogAndFlush(string clientMessage)

       HttpContext.Current.Response.ContentType = "text/plain";
       HttpContext.Current.Response.StatusCode = 500;
       //lets stop everything here so only HI will be sent
       //code line to stop response

so lets see our options:
1. no extra code - the client gets "HI OK TREVIL", meaning that i clear the response, but the response bubbled to the main T/C, and continued to the code after it. in the eventviewer i get "OK TREVIL", where i want to see nothing at all.
Conclustion: i could always let the error bubble and check for EX2 and skip hadling (double catch).
in a Page: also require flags.

2. Response.End()  - the server is good, since no event is getting logged, but my client gets "HI BAD", and while debugging i saw that this is because the ThreadAbortException was thrown and the main catch cought it.
Conclusion: if i accept using a ThreadAbortExeption handling (double catch again) i get just what i wanted although its considered bad practice.
in a Page: same.

3. HttpContext.Current.ApplicationInstance.CompleteRequest() -  so i googled to get promted to use CompleteRequest(), there i get again "HI OK TREVIL" and "OK TREVIL", so there is no meaning!?!
so after some googling i found this bit more detailed stack
where Jay Zelos informs about Response.SuppressContent = True
so i replaced my code line to
Response.SuppressContent = true;
the result in the client was great, only "HI", just what i wanted. i debugged for safety to find out that the code was executing as usual to my unhappy suprise, and in my sample code i got in the events "OK TREVIL", meaning that the EX is bubling and the rest of the code is executing.
BTW removing CompleteRequest() give the same results.

Conclusion: - Flush() + SuppressContent does the job client-side-wise, and leave me to handle my code. with a double catch for my EX2 i'm preety good.
in a Page: same (must be used with Response.Write())
Second Conclusion: seems that CompleteRequest() for us currently dont do much more that performance...unless you have something in modules that anoys you.

optimally in my case was using both SuppressContent and Response.End() with a double catch for ThreadAbortException.

Undestanding the difference between Response.End() and CompleteRequest():
the theroey is simple. every request to the IIS is initially wrapped in an HttpApplication object, that has a set of events that when fired can be implemented in HttpModules, there you are still in the context of HttpApplication as a sender, although you still have as a property your HttpContext.

but then each request find its handler (only 1), and from there on your context is HttpContext object as a parameter. from that point there is a hole new chain of events and when they end the request returns to the upper chain of event. you cannot modify anymore the HttpApplication, only access it.

CompleteRequest() says that the handler/page finished the job and the modules dont need to continue their jobs. it trusts the coder to handle everything inside.
Response.End() is a knife cutting everything, it kill the thread, and as MSDN states it was made for a time and scopes with no better solutions.

and if i want to implement a "safe" Response.End()
marry for us many people did a reflection on Response.End() and here is how its implemented (taken from the stack Q):
public void End()
    if (this._context.IsInCancellablePeriod)
        Thread.CurrentThread.Abort(new HttpApplication.CancelModuleException(false));
    else if (!this._flushing)
        this._ended = true;
        if (this._context.ApplicationInstance != null)
we can see 2 very interesting things here.
the second is that if the context is not cancellable and the response is not yet flushing it implements just what i did in my handler, flush, i assume that this._ended is somewhere equals to SuppressContent, and if its inside an HttpApplication, CompleteRequest(), the "recommended" behavior.

what shines for me is that it just aborting the thread as is, so i tried that, i just replaced my line with Thread.CurrentThread.Abort(). the results was devastating, the IIS pretty much stopped working, many errors were logged, and no response at all in the client nor server (with the IISExpress he was closed).

i looked for this HttpApplication.CancelModuleException but its a sealed class, not for humans. MS has their own way to kill a thread without killing the IIS, and they are not sharing (wrapping in a T/C didnt help).

bottom line is that ther is no replacement for Response.End().
a right code pre-handles a case for stopping the standard execution, similar to the link above and can help itself with Flush() + SuppressContent + RequestComplete().
a short code uses the Response.End() with T/C.

for deeper understanding read this great presentation

Monday, July 7, 2014

Create Handler tutorial and example

creating a handler is a super simple task - if u know what that task needs

there are 2 options to create the same thing.

right click and add new item from your visual studio and there will be 2 options, one named "Generic Handler" and the other "IISHandler", both will create a .cs file and a class inheriting from IHTTPHandler.

the Generic Handler also creates an ashx file, which will save you another line in your web config, so after creating ur generic handler go to ur web config and find the <handlers> tag and add the following:
<add name="myHandler" verb="*" path="myHandler.ashx" type="myNamespace.myHandler" />

in case you are working with GAC than you must add to type the assembly info like this
type="myNamespace.myHandler, myNamespace, Version=, PublicKeyToken=71e9bce111e9429c " />
of course that the data i put here is totally random, so learn how to get yourself ur real assembly info

no in case you created an IISHandler you wont have a .ashx file and that's cause its made to be kinda like a service, build it and then create and configure it in the IIS.
but you dont have to, just register it in your web config under <httpHandlers> in the exact same way, and the exact same line.


Thursday, July 3, 2014

sharepoint 2013 - cross site publishing for the anonymous site

you can have it in 2 ways.
the easy one is to extent your web application and basically all problems solved, since you have all ur data with an anon face.

but if you want to have 2 different site collections, 1 anonymous for the public facing and another dedicated for the catalog(s) then there is a catch - for the items themselves of the cataloged list you can leave the anonymous access and everything will be alright, but if you use the dedicated site's resources than ur in for a login promt. and giving the anon users lists and libraries only doesnt fix that, so ur in for 2 anon sites.

for example i have 2 catalogs lists, one have its images as links and secondary as images from an image library. the 1st list items are shown in the public site, but the 2snd list items are not unless i give it full anon access.

and dont forget - you must also cancel the break inheritance in the master pages gallery, and either publish all the new stuff or cancel versioning (the publish/version is everywhere actually)

Wednesday, July 2, 2014

Sharepoint Connect to Catalog Error

Terms can not be shared multiple times in the same term set.

i thought mine was because the Metadata Column wasnt really connected to my Term Store, and that caused that i chose terms existing in the term set yet he didnt find them and tried to recreate them.
all that started cuz i didnt check the Available For Tagging checkbox.

note that he creates teh pages anyway again and again.

then i just deleted the terms i originally put and it worked, but now i am without terms...

so something is just wrong and i'll go find myself a new tutorial

after some extra play i think i got it
if in your navigation settings you set for structural navigation then you get an error: "We could not find the navigation term set for this site. Navigation integration and friendly URLs will not be configured"

so to fix that you need to go back to navigation settings and set the navigation to managed navigation. but there if you choose the term set that the catalog is going to use (cuz its the only one there cuz its a test site for testing the catalog feature) then he "tries" to double set the term set.

eventually i chose on of the OOTB empty TS and connected again and i made it :) !!

Sharepoint Search Crawl Exception, Search not crawling documents

The content processing pipeline failed to process the item.
( The type initializer for 'Microsoft.Office.Server.Search.LocStringIdLookupTable' threw an exception.
 SearchID = C7F1F2A7-1EF3-4D78-A0BF-E93A752C6236 ).

mine was because i limited the noderunner.

C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\noderunner.exe.config
set <nodeRunnerSettings memoryLimitMegabytes="0" />

anyway for these kinda problems restart the search service and iisreset

Tuesday, July 1, 2014

Sharepoint Anonymous Search Rest not working

status 500, Microsoft.Office.Server.Search.REST.SearchServiceException, unknown error occurred.
if that's what you get, then go take a look at the ULS logs, there is a lot of info there.
one of them might be "An XML declaration with an encoding is required for all non-UTF8 documents." that mean that the queryparametertemplate.xml is encoded with something that is not UTF-8, so download it, and save it again as UTF-8 (preferred to delete the old copy from all recycles before)

See Database (SQL / WCF) data in Sharepoint - Implement BDC and Search

Scenario - we have an SQL DB with a lot of data, and we want to put it in a sharepoint site, and even more we want to show it to our Anon user (wwwhhooooooo!!!)
*this article was tester in SP 2013 but is valid for 2010 too.
*BDC = Business Data Connectivity
so minimal background:
1. Anon users cannot see anything but pages. we shall see how to overcome this.
2. BDC is a magic tool that with it you can introduce you database to sharepoint and teach them to play toghether.
3. you can search and index the BDC data.

conclusion: we want to index our BDC and with Content Search WP or Rest show the data to the anon users, so lets go:

1. Start the services:
we must make sure that the sharepoint BDC and search service are up: Central Admin -> Application Management -> Manage services on server:

2. Provision service applications:
the fact that the services are now up and running means that there is a platform ready to use, we now need to make instances,  Central Admin -> Application Management -> Manage service applications, click New and provision those :
*if you have instances of those services you may as well use the o.c.

now go and click the BDC service and inside click on the ribbon click Configure, and put there you host web, i.e. your lovely website domain (http://mysite).
in any way, make sure your new services apps. are connected to you web application, Central Admin -> Manage web applications -> mark your web app -> on the ribbon 'Service Connections', if your new services apps. are not there then choose custom and rebind.

3. Create External Content Type:
the way SP see your data is in the only way he knows: lists and columns. so we give him that exactly by using SharePoint Designer, rather easly btw - open the Designer and open your target website (if you didn't have a dedicated one by now, time to create it :D )

*REMINDER - any user that should contact the external data should have permissions there, so for example your SQL DB should give permissions to the search (sp_searchscv) and iis (IUSR). so if u didn't do it until now, its a good time.

there you have on the left nav. click External Content Type, and then on the ribbon top left click External Content Type. give it a nice name. then click on the most bottom link that says 'Click here to discover external data sources and define operations'.

click 'Add Connections' and connect to you DB, choose a table or view, right-click and 'Create All Operations' (unless you cant for some reason).

click Next, and choose the Key columns and check the 'Map To Identifier' checkbox and Finish. you can play with the Display Names of the columns as needed.
Save your work.

if you want to see private items in ur web then click on the ribbon Create Profile Page (u wont be able to browse them in anon, so if its purely for anon u don't have to).

BTW you can do just the same with a WCF, the only difference will be that you will have to set each operation explicitly, it works great.

4. Setting permissions to you External Content Types:
back to Central Admin -> Application Management -> Manage service applications -> you BDC -> you should see your new External Content Type, if you hover it u'll get a little drop down and click set permissions. I think that the real user that will activate it in the end will be IUSR (the IIS user) but I am not sure, I just added All Authenticated Users.

in case you want to see it in action, you can create external lists in your web, activate web feature ''Team Collaboration Lists" (רשימות של שיתוף פעולה בצוות), then create new external lists based on the new content type and see it in action.

5. Indexing all that:
Central Admin -> Application Management -> Manage service applications -> you search-> Content Sources -> New Content Source :

*make a new content source for each table, since there is a problem (bug?) choosing Crawl all, and I think its a 2013 only thing.

but id u'll crawl it  wont have any managed props to retrieve. go to Search Schema, New Managed Property, and create one for each of your colums. a nice trick by Jorge is to make a prop with all the cols for ease of search. might be a bit of a hard work to create a new Managed prop for each prop, but you can always add them to existing managed props.

and...start a full crawl :). check the log that you got no errors, and if you do its usually permissions problems for sp_searchsvc.

6. Put it on a page:
so lets put it in a Content Search WP somewhere. the easiest way for me to get all the results for this specific content is to go to my search service, click search schema, find Content Source, and set is as Searchable, so I can put in my query ContentSource:MyContentSourceName.

i'll set the CSWP like this:

7. See it in anon:
now publish that page and open it in the anon site (and in an anon browser - make sure u don't see the ribbon)

8. Customization:
for customization we have 2 options, 1st is to learn how to query that with the Rest, and then we can use it anywhere (say with angular :D ).
here is a easy tutorial with a PS for it
if you get a 500 than make sure the .xml file is saved in UTF8 format, or check ur ULS logs.

var url = http://yourAnonSite/_api/search/query?querytext='ContentSource:TestTable1_WCF'&selectproperties='TestTable1IntCol'&queryTemplatePropertiesUrl='spfile://webroot/queryparametertemplate.xml';
jQuery.ajax({  url: url,
    headers: { "accept": "application/json;odata=verbose" }

as search is, the data is in a path like this usually : "d.query.PrimaryQueryResult.RelevantResults.Table.Rows.results", which will give you and array, with each one "results[0].Cells.results" is another array for each colums (well, managed property)

the 2nd is to use Display Templates:
well, to be continued...

That's All Folks!!

u also might need this PS:
$bcsServiceApp = Get-SPServiceApplication | where {$_ -match "ur bdc SA name"}
$bcsServiceApp.RevertToSelfAllowed = $true;