Months ago I upgraded our production farm to SP1, more specifically FEB 2012 CU, coming out of that we considered the upgrade successful in many ways, however something that was not expected were issues with our User Profile Syncs. Here we thought we were improving the User Profile Service by upgrading to the newer CU. I spent many days after the problems started occurring trying to figure out what was going on and so much time invested to the point where it was blurring my vision on what was really going on with it. I was basically wrapped up in if our syncs were actually working or not…were they changing information in the profile store when AD changed and were things really working right even though I was getting errors. Here are the errors I was getting each time I tried to do an incremental sync or a full sync. This all started with a full sync…when I kicked that off POST upgrade to resolve something else, it started this series of errors and wouldn’t go away no matter what type of sync I ran.
In my environment we leverage SharePoint to feed profile images from the profile image store over to AD so that Exchange can leverage them in Outlook. The odd thing about this situation is that you would think that if I was getting these errors that none of my image exports would be working right? Well I thought the same and once I found that they were actually working I realized that this set of errors may not be as impactful as I thought…So my lead and I decided to table it for a while since we didn’t have any one complaining about issues and things looked fine even though things were erroring out during our syncs. Frankly I figured that it was to impactful to our operations to rebuild UPS in order to possibly fix the problem, when it may not fix anything at all and could create more problems.
Present day….working with my development farm today. Something I was doing for another project compelled me to look at this problem again. So I started digging around but with a clear mind on the subject…I found an article that I had come upon when originally troubleshooting but hadn’t looked over it completely…I knew it related to the profile images and possibly the path but not much more…but in this article an idea was formed.
The article was written by Henry Ong : Case Study on Troubleshooting a Failed SharePoint User Profile/FIM Synchronization, Bad CMYK User Photos and Disappearing User Profiles
He describes a similar scenario in which I believe he ends up recreating his whole UPS service DB and all, but what I over looked in this article was the comment stream! There was a post by Darren Martin who suggested running a SQL Query to identify which images had corrupted URL’s…Darren also goes so far as to suggest running another update directly into SQL from what I could tell, to replace the bad values with “NULL”. Now at this point in my young career with SharePoint, I’ve read to many articles that mention direct to SQL is “bad news bears”…so I was hesitant to consider it. I wanted to at least run the query against my profile DB though to see what popped up….
SELECT RecordID, NTName, PictureUrlIn the results I found that there were several profiles that were pointing to an image path of an active test user account we have, but when I looked into the profile image library at the image they were referencing I could not find it. So this helped me understand a bit more about why the errors were generating…from what I could tell FIM was looking through all of the paths and validating them against the profile image library. So I uploaded an image for the test user and then ran a full sync against my UPS…SUCCESS!! No more issues! I later found out from my lead after letting him know what had happened, that before I had come on board a request came in and he was asked to modify all the images that were generic for a users profile to a standard image….in his attempt to do so he created a timer job that looked for value “NULL” and replaced it with the test user account image….however he ran into issues with it and abandoned it. However the settings remained in place and I assume since SP1 brought new improvements to the UPS services it wasn’t caught until then. In the next chapter of this saga I’ll be looking into how to safely revert this back to “NULL” for those accounts affected, as they will still need that test users profile image to stay put in the profile images store for now, to keep our syncs working correctly!
Found an article by the SharePoint Blues Team in which they described the exact same problem I had, they provided a script which I have linked here for safe keeping which will go through all of the profiles and set them back to NULL for the picture value if the URL cannot be reached.
Link to Script
At this point I have not tested this because I'm building a 2013 farm but if it comes up in my 2013 build i will be trying it out...if you use it and are sucessfull (or not) leave a comment to let others know!