Invalid or missing properties when saving an Office document

Sometimes you try to save an office document in SharePoint and it decides that it is missing some properties in something called the 'Document Information Panel'. This panel appears at the top of the Office client (Word, Excel, PowerPoint, etc) with the columns you have created within a document library for users to fill out key information ABOUT a document (often called document metadata) so that you can find that document in the future by using those columns.  Here's the thing:  when you see this error about missing or invalid properties - it usually means you filled out something incorrectly or there's something you forgot to fill out...but not in my case.  In my case, I did a bad, bad thing and created a column called DocType.  NEVER EVER CREATE A COLUMN WITH THE NAME OF DOCTYPE!!!!  It will immediately give you an error aftter you create it that it cannot get the ContentTypeID and will never allow you to edit the column again.

So, I decide that I'm going to fix it by just hiding the column. Well, that makes it so that I get the lovely 'invalid or missing properties' error but then there's no property that I can actually fix! I found a post here:  http://www.novolocus.com/2010/05/10/to-save-to-the-server-correct-the-invalid-or-missing-required-properties/ that shows that you can inspect your document within Office to remove Custom XML data and that that should fix the problem...it lets me then save the document. YAY! BUT....then I can't check the document back in.  So, I HAVE to leave DOCTYPE as an optional column and just tell users not to fill it in.  Moral of story - do NOT create a DocType column or you will see the following error messages:

'Object reference not set to an instance of an object.   at Microsoft.SharePoint.ApplicationPages.BasicFieldEditPage.get_ContentTypeId()'

'To save to the server, correct the invalid or missing required properties.'

Folders, Metadata, SharePoint 2007, and SharePoint 2010

So you heard the news:  folders = bad in SharePoint land; but why?  We have spent the last 20 years putting things in electronic folders, the past hundred putting them in real folders in real filing cabinets....so why does SharePoint come along and say "folders:  the dodo was better"?  Well, let's talk about the reason we had folders, why we have them now, how they work today, and what can be done to improve the situation.

Physical Folders

So uh, a long time ago...people used paper.  Paper was this collection of ground up trees in a paste that is super flat and thin...when it dries, it's nifty to write on.  Writing is when we use a semi-permanent or permanent utensil to make marks on this paper...we would write what we now type...don't worry, you won't need to remember this.  When a bunch of this "paper" was used to gather information, they needed a way to organize it so that you could retrieve the information later (whenever someone asked or sued for it).  They decided they would use thick paper and bend it in half to hold several pieces of thin paper inside it...we would then mark it with useful information so we would know what was inside it.  We would stack these things called "folders" together, sometimes put them in big drawers called filing cabinets, and that was a great system till people realized that paper - being made of trees - was flammable and permeable by water....and, on top of that, it takes up a LOT of space.  Introduce electronic folders.

Electronic folders

So, we fast forward to the stone age when computers first began.  For a reference on how the first computers worked, google "The Flintstones" and try to find an old video that shows you this animated entertainment clip of life during this time.  We decide that paper is a thing of the past and decide that typing is the way to go (hooray for lefties around the world!  our handwriting is no longer an issue!).  We type up papers, resumes, and all sorts of stuff.  So, we have all these types of documents we type up for work, school, presentations, reports, etc...and the problem came back:  how do we organize this stuff?  We decided to transform a single, magical folder into the digital world where it replicated itself as the first virus to spread across the digital globe.  People started putting documents in these digital folders to help "organize" information.  Do you see the problem yet?  Still bunches of folders, still bunches of stuff, still dealing with a mentally-deficient chimp's method of organizing documents and you aren't allowed to change its methods.  People figured, "hey, this is digital...let's be bold and put folders INSIDE folders for better organization!"  Oh yea, we were that dumb.  So, going to find stuff really was just like regular folders:  took forever and you were at the mercy of who (or what)ever setup your folder structure.  We made really smart searching tools to let us peek inside folders faster, but the problem remained.  Bring in Metadata.

Metadata

The first concept of metadata is quite cool:  let's take all the important information ABOUT a document, and stick it somewhere so we look at THAT first instead of the folders.  We look through a stack of metadata really fast, find the paper we want, and it tells us where to find that paper.  You got it:  a catalog system...you know, like libraries....the place where they would but books....which were collections of paper bound together and forced you to read ALL of it to find that perfect page....yea, no iTunes for book chapters or phrases :(.  Anyway, this new metadata idea lets us keep our folders and all our stuff wherever - we just need to take a few notes about the document before we file it away in the black hole.  Doctor's offices use this a lot - they have color codes, name codes, and tons of shorthand written everywhere...so much so that there's a job and training just to decipher it!  The cool thing is, once you know the code, they would pack thousands of people's medical records into a space the size of your family room.  We would use this on electronic documents to make searches "faster".  Now comes SharePoint.

SharePoint

So, this metadata thing sounds cool, right?  Well, what if you end up in a doctor's office that held 100,000 people's information?  You'd still be looking at a lot of the same problem because you would have a LOT of metadata and THEN you'd have this massive mound of stuff to look through to find your document - even though you know where to look.  So, the way to solve this?  Instead of using folders - which HIDE information inside them till you look there - let's put everyone's information in giant stacks with the metadata sticking out on each document...and let's pretend that we can control gravity on each document so that, when we say we're looking for a purple tab, all the purples come together RIGHT in front of you.  Nifty huh?  That's what SharePoint wants to do for you.  In 2007, they let you make folders.  The only real reasons to make a folder are to separate stuff that needs unique security permissions (locked down folders) or if there's more than 5k items...then use folders to break down some of it into really big chunks...maybe.

The problem:  most people didn't know about metadata and still made electronic folders.  The SharePoint gods became disturbed and rained fire from Mount Despair on all such places - making them slow and difficult to use and making people forget that they do this everywhere else but SharePoint so that they complained to the Admins..err High Priests of SharePoint that they didn't do their job.  So, the evangelists of SharePoint are now proclaiming the wrath of the gods on folders...please, don't use them so you won't be burned.  You can choose not to believe in the SharePoint gods or the warnings of folders - but beware, SharePoint gods don't believe in folder athiests.  Use COLUMNS to ask for important information about documents and items when they go into lists and libraries...have 3 or 4 columns instead of folders.  You can search and filter by columns!  You can find whatever the heck you want super fast!  So, how has it changed in 2010?  NONE!  The only thing they did to make it easier on you folder lovers was to make it so that anything inside a folder got a special tag on it with a piece of metadata from the folder.  This way, every item automatically has some metadata - so poo on you if you don't want metadata, it WILL be branded on every item!  May the SharePoint gods smile on your learning; go, young padawan, and folder no more.

InfoPath Tutorial updates

Hey everyone,
I just wanted to let you know that I've updated the Introduction and Part 1 of my InfoPath 2007 Tutorial.  I'm sorry it took so long (over a year) to update them...work has been crazy and we've been learning so much about InfoPath and SharePoint 2007 that it's been scary...then we decided to push on to SharePoint 2010 so much of my time has been learning and planning on our 2010 setup.  Once I finish my 2007 tutorial, I'm going to put out a series on InfoPath 2010 so that you will have something for either program.  One thing I do have to say:  you can use InfoPath 2010 to make 2007 browser forms...and dear Lord if you have the opportunity to do that...TAKE IT!  There are a few major improvements in InfoPath 2010 that can be used to drastically cut the amount of time it takes to make a 2007 form.  I look forward to releasing InfoPath Tutorial - Part 2 - Color Schemes and Controls next week.

InfoPath Tutorial - Part 1

SharePoint Permissions Architecture - Part 1

SharePoint is a bit of a beast when it comes to managing access to sites, lists, and libraries.  To plan out many of these involves discussions on every major layer of a company and their various needs.  The building blocks of a Permissions architecture are the following:
1. Users -> Users are defined using Active Directory (a server that is like a giant contact list for your company...it holds who you are, what dept you are in, your password, phone number, etc.).
2. Active Directory Groups -> In Active Directory, groups can be made so that you can work with people as chunks instead of individuals.  This is a great help when it comes to thousands of people being involved in your company...who wants to manage them all individually!?
3. SharePoint Groups -> Active Directory groups are more common as larger groups of people.  At a school, for instance, there might be groups for Faculty, Staff, and Students (amongst others).  There might even be groups broken out by department.  Often there is a need for more specific groups when it comes to sharepoint or even a mixture of multiple groups.  So, SharePoint will let us create our own groups of people (and you *can* include AD groups INSIDE or AS A PART OF a sharepoint group).
---
4. Types of Authentication - what does Authentication mean?  It means how your company verifies your username and password so that only you can access what you are supposed to (keeps other people out of your business ^_^).  There are several types between SharePoint 2007 and 2010 but these are usually set by your company's IT department and not something you generally have control over.  A couple of examples include:  Forms-Based Authentication (whenever you try to go into SharePoint or Webmail or some other Microsoft-based stuff, you get a webpage that asks for your username and password and then sends you to the webpage you were trying to go to) and Windows Authentication (pops up with a login box for you to type your username and password - and often a domain - and it logs you in and takes you to your page...though, you often have to log in again if you try to go to another system like Webmail, depending on how your company has set things up).  2010 introduces a new type of authentication called Claims-based authentication, look it up on google for a fun read, but is sort of a hybrid of multiple authentication types.
---
5. 3 layers of access in SharePoint - there are 3 primary places where permissions can be set:  the Site (giving access to an entire webpage), the list (you can set specific permissions on all the lists or some of the lists in a particular site), and the item (items INSIDE a list can have their own permissions set so that certain people can't see certain items...great for manager-only items in a public location but SHOULD BE AVOIDED if at all possible). 
6. SharePoint Permissions Inheritance - SharePoint's permissions are connected and automatically set (and updated) by the level above whatever you are looking at (see the levels in the previous point).  If you make a change at that uppermost level, all things underneath it reflect that same change and you are only allowed to edit the Site level.  This works great in an ideal world - and you might be able to get some leverage out of this on occasion, but there are too many reasons to customize permissions for a certain list or library.

So, that's it for now, the next post will explain how to arrange your building blocks into a decent permissions standard for the organization.  Mind you, there is no one-size-fits-all solution for permissions architecture but this one will get you started thinking along those big-picture lines of permissions.

InfoPath Form Layout - Proper Design

So,
You've decided to create an infopath form but don't want to do the whole "let's make this and see how it works and then figure out the 'right' way to do it" approach. One simple tip that will really help in laying out your form for both presentation sake and for data source/structure's sake: EVERY label AND control should have their OWN cells inside a table which is inside a section which is inside another table.

This may sound strange, but let's break it down further. You have a simple form with 4 fields (First Name, Last Name, Favorite Food, and Favorite Color). You should firstly think of how the information should be grouped (name info and favorites info) and use that to think of how many sections you will need. For this we would need two sections. Once you have your number of sections, add 2, and that's how many rows you need for a simple form (one of those rows is the title row and the other will be for the submit button). To create this layout, I used a Table with Title layout table, split the second cell into 3 rows, removed the shading on the second row and all borders on those bottom rows, gave it a title, put two sections in their respective rows, and put a button down at the bottom. From there, you place a Custom Table inside the sections to allow you to put controls and their labels into the section in an organized fashion. Having done that, name all the controls and sections by double-clicking them and changing the Field or Group Name. Here's what the results looked like in both the table and then the data source menu:





This particular way of doing things will keep all of your data sources in named folders and makes it easy for those coming later to identify where a data source should exist on the basic page. All corresponding data is automatically grouped together for easier retrieval, if necessary, from them pesky programmers. Have fun!

So I've been gone a while

Well,
It's been a while but I'm starting to come back around to updating with some new posts, so here's some of my recent stuff:

1. Our SharePoint setup was such that everything was in a single content database and single site collection. This has proven troublesome, especially in MOSS 2007, so the system admins are going to split some subsites into their own site collections and group them into multiple content dbs. We're hoping to be able to point out where some of our current performance flaws exist in doing this but also to prepare for a migration to SharePoint 2010.

2. Working on the CTT+ certification...fun times there...pro tip: don't have people use funny first names in your video - creates a world of havoc in submission and processing.

3. Created a vehicle registration form for the university to register (2) vehicles in an infopath form, submit each vehicle as a separate form to sharepoint, and then process those via Access on the backend. This is a pretty simple but cool setup and didn't involve any major work EXCEPT: infopath decided, upon one of the publishing processes, to add a 'comment' tag in 3 different places...there wasn't a comment opening tag or any actual comment, just that closing tag. I also got an error for "unexpected CSS formatting" or "potentially unsafe HTML found in your form". To fix the comment issue, you have to do the scaries thing ever: Save your infopath form "as source files" and then open the particular view.xsl in notepad++ and look at the CODE...scary...and then do a simple find for the word 'comment' and see what you find. I just deleted the tags since they weren't connected to anything and that fixed them (you won't see them anywhere in the form, just in the code).

4. Learned a nifty trick on Microsoft's website for determining if someone who is opening an infopath form is opening it in the browser of if it's being opened in InfoPath: use a condition on an opening rule that says "matches the expression": "xdEnvironment:IsBrowser"...this is useful if you want to have certain views used internally for processing but the main page be a browser form for public use.

5. Had an interesting problem sent to me: someone deleted a list in sharepoint and, when it was restored (not from the recycle bin), the list had a new ID number (called a GUID). This messed up several things that were originally connected to this list; however, there was a simple albeit scary solution for it: open up the site in sharepoint designer and just do a simple find-replace for the original list's GUID with the new one...had several thousand fixes - mostly from workflow configuration and workflow history. After I did that, I double-checked the workflows themselves and noticed that you can open these up as xml files and just do a find and replace there too - so i did. Between the two pieces, it fixed all references to the list and all workflows now work just fine.

SharePoint KPI Best Practices

So, I'm hoping people will see this in google and give me some ideas. I'm trying to figure out some best practices for KPIs in MOSS (SharePoint) 2007. Thus far, it seems best to explain to those new to the topic why I'm looking for this.

A KPI stands for Key Performance Indicator, which basically means this is a report-at-a-glance of something specific in SharePoint. The more you track in SharePoint, the more you can generate these quick pictures that highlight (high-level) how areas are performing and give management a way to analyze the health of their departments, teams, divisions, etc. An example of this would be a task list:

1. You have a task list in SharePoint that shows every task that your team is working on right now and in the past.
2. You are given the responsibility of figuring out how many tasks are over a week behind schedule for every team member. Now, you could do this by sticking a bunch of the data in Excel and then using filtering and conditional formatting to get your data...but that data would be flat/static and you'd have to do a little extra work to make it dynamic...even then, it's not dynamic without a refresh. So, you want to make a KPI.
3. To make a KPI, you use the Site Actions->Create menu and create a KPI list (this is only available in MOSS 2007 and with the "Office SharePoint Server Enterprise Site features" is activated on your site). Form there, you can create a KPI based off of a SharePoint list and then set it to the URL of a view and then either count the number of items or a percentage of items exist there (even add additional filters for percentages if there's a number column involved). You then set whether a higher or lower number is better and what numbers change from an unsatisfactory to "ok" to satisfactory. This KPI list can then be opened directly or else placed on a web-part page in a dashboard style.

This is a common scenario and you may need to report quite a bit of information off of the exact same data. So, let's look at some performance best practices for KPIs:

1. SharePoint KPIs are based off of list views which means that the KPI will only be as good as your view. Views can be optimized by remembering the 3 F's: Folders, file indexes, and filters. Create folders to separate data (but don't let everyone create folders, make them yourself). File indexes involve taking a specific/key column in your list and having SharePoint make a copy of just that column so that it can reference it when searching the list (in views and, by extension, KPIs). You can turn on an index for a column by going to the List Settings and looking under the Column Section for "Indexed Columns"...just remember not to index more than 1 or 2 columns. Finally, filters will help your views keep from retrieving too much data at once (which would then be interpreted by KPIs). Use filters FIRST on indexed columns and THEN on non-indexed columns. Make sure that no view, personal or public, would retrieve more than 2000 items at once. This isn't a hard-and-fast requirement but it's best practice for SP 2007 (2010 raises this limit significantly as it handles your views/queries a little differently). Also, even if you only see items in batches of 5, you can still be pulling a massive amount of data (you just have it set to display them in batches of a certain number)...so verify the number of items returned when you switch to a certain view.

2. KPIs can be generated through a KPI list and also through calculated columns/data view web parts (see www.endusersharepoint.com for more details on the latter). Where this is displayed is a key performance difference: if you can display a KPI inside a list (as a column) then you will be able to maximize performance. The reason that this can maximize performance is that each KPI queries a list view...which means that 10 KPIs that use the same view but show slightly different information will re-query the same view 10 times. If you have a column built into the list then you can include a KPI as a part of a specific view (maybe a reporting view).

3. If a KPI collection is going to be on a special page (maybe a dashboard for management), then there are some factors to think about: the number of KPIs, the views associated with those, and any custom graphics that we're trying to display; all of these will be performance hits on your web part page. To minimize these, try to actually use a list form web part and show a specific view of information in a list whenever possible. This can be a specific view of all late tasks grouped by person instead of a KPI that reports that information (so you don't get a pretty icon stating whether or not there's too many tasks behind schedule, but you'll see a number and be able to see corresponding data too). If we have more than 10-20 KPIs on a page then it takes maybe 30 seconds to display the data correctly (though some of that will be through inefficiencies in our own organization), so use KPIs to seriously high-level ONLY the information that managers need to see in a web part page and RESTRICT ACCESS to this page to the specific manager or manager SharePoint Permission group so that the page isn't constantly refreshed.

I'm welcome to receive other suggestions so that someone can reference this information as we discover more about KPIs. If any of this information is incorrect, please comment and explain why. Hope this helps!