Skip Ribbon Commands
Skip to main content

View blog authority

Home
This is my website for sharing my knowledge of SharePoint and other Microsoft technologies with the world.
 

 Most Recent 3 Posts - For All Posts, Click Here

 
Using PowerShell and the .NET CSOM to Query SharePoint 2013 Online

 

Hello from the Microsoft SharePoint Conference 2012!

One of the things I wanted to do while here was learn how to use PowerShell to interact with SharePoint 2013 remotely.  Now remotely can mean two separate things here: 1) Using PowerShell remoting to send POSH commands to a server to be run locally, or 2) using remote APIs (in this case the .NET client side object model, aka .NET CSOM).  Since I want to have POSH that can work with both SharePoint on premise and SharePoint online, I decided to use the .NET client side object model.  I was developing against a free developer instance of SP2013 online, which you can request at http://msdn.microsoft.com/en-US/library/office/apps/fp179924.  At first, I tried using the example on how to retrieve a list that I found here: http://msdn.microsoft.com/en-us/library/ee538683.aspx After converting it to POSH, it was:

$loadInfo1 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
$loadInfo2 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
$webUrl = Read-Host -Prompt "HTTPS URL for your SP Online 2013 site"

$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($webUrl)
$web = $ctx.Web
$lists = $web.Lists
$ctx.Load($lists)

I ran it and got this error:

Exception calling "ExecuteQuery" with "0" argument(s): "The remote server returned an error: (403) Forbidden."
At line:11 char:1
+ $ctx.ExecuteQuery()
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException

That kind of made sense, as I wasn’t passing any identity information, and the Microsoft Account associated with my Windows 8 domain account was not my SP Online Microsoft Account.

I looked and looked on MSDN, and didn’t find much helpful.  I tried setting the $ctx.FormsAuthenticationLoginInfo but that didn’t work.

Eventually, I gave up and decided to catch up on my reading of the Product Team Blog, in particular this post: http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?pID=1048

At the end of the post is a PowerShell script that passes SharePointOnline credentials!

$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)

So now, I included that in my script, and I have:

#This works on SharePoint Online sites!

$loadInfo1 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
$loadInfo2 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
$webUrl = Read-Host -Prompt "HTTPS URL for your SP Online 2013 site" #"
https://michaelbl.sharepoint.com"
$username = Read-Host -Prompt "Email address for logging into that site" #michaelbl@michaelbl.onmicrosoft.com"
$password = Read-Host -Prompt "Password for $username" -AsSecureString


$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($webUrl)
$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
$web = $ctx.Web
$lists = $web.Lists
$ctx.Load($lists)
$ctx.ExecuteQuery()

$lists| select -Property Title

which outputs:

Title                                                                                                                                                  -----

App Packages 

appdata

Apps in Testing

Composed  Looks     

Content type publishing error log

Converted Forms

Documents

Form

Templates

Get started with Apps for Office and SharePoint

List Template Gallery

Master Page Gallery 

Project Policy Item List                    

Site Assets

Site Pages       

Solution Gallery

Style Library

TaxonomyHiddenList

Theme Gallery

User Information List

Web Part Gallery

wfpub

There you have all the lists in this particular SharePoint Online (Office 365, SharePoint 2013) site!

--Michael


# Comments:
Modified: 11/15/2012 1:25 PM
Permalink: http://blumenthalit.net/blog/Lists/Posts/Post.aspx?ID=171
SharePoint Fest 2012 Recap

 

I had a great time at SharePoint Fest Chicago this year! It started off Tuesday night (September 25, 2012) with our CSPUG (Chicago SharePoint Users Group) meeting. We had over 100 people register and most of them show up (every event always loses some attendees between registration and the event). We had a decidedly full room in the Stephens Convention Center in Rosemont. I had the honor of being the emcee for the evening. Our panel consisted of Brian Culver, Todd Klindt, Sean McDonough, Gary Newman, Dux Raymond Sy, Randy Williams, and Jeff Willinger. The event was recorded, and I’ll tweet about it when the video is available online.

The next day, Wednesday, I presented my Introduction to PowerShell in “SIA 100 - PowerShell with SharePoint 2010 & 2013”. You can use PowerShell to work with MOSS 2007 too, but you don’t have any pre-defined cmdlets to start from, so you need a script like define-sharePoint-functions.ps1 that you can get from my CodePlex project: http://psbb.codeplex.com/SourceControl/changeset/view/20689#259920

Fortunately, all of my audience was running SharePoint 2010, so they get to use all the 500+ cmdlets that ship with the product, as well as any they download or write. My slides for the session can be found in this folder:http://blog.blumenthalit.com/files/spfest2012chi.  It’s the PDF called PowerShell for SharePoint Developers and Administrators.

I did a brand new demo in that session, which I call “WebParts On The Bus”.

So far, it consists of three scripts:

  1. Create the “Bus” site collection using the Blank Site template (STS#1), and open IE to display it.
  2. Use my Make-Flashcards script to create images for the site’s home page.
  3. Rotate which web parts are in which web part zones, so the web parts on the page go round and round, and make the computer sing along.

There’s plenty of room for further automation, such as uploading the images, creating the web part page, adding the Image Viewer web parts to the page and configuring them to use the right images.

Also, only the third of these three scripts is compliant with my own best practice of being written in a way where running the script file is harmelss and error free by default.  What I mean by this is that script #3 only defines a function and announces the function has been defined, but does not do anything unless you call the function.  The other two scripts have lines of powershell that would be executed when you run the script.  For example, the second script defines the Make-Flashcards cmdlet, but also tries to make flashcards, saving them into a folder that likely does not exist on your machine. Don’t run them until you’ve read them and adjusted them for your environment.  In other words, use all three AT YOUR OWN RISK, no warranties expressed or implied, et cetera, et cetera…

The scripts can be found in http://blog.blumenthalit.com/files/spfest2012chi.  A number of other PowerShell scripts can be found in http://blog.blumenthalit.com/files/powershell%20scripts, but again, use at your own risk.

After I gave my session, I walked around the expo hall, and met with a number of new and old friends.  Here’s a quick rundown of the expo hall highlights, focused on the ISVs.

  • K2 was there, with Chris Geier at the booth.  K2 provides workflow solutions.  I’m sure that oversimplifying, but my primary goal was to learn about the vendors I hadn’t heard of before, and secondarily to talk to the vendors I already knew.
  • Newsgator was there.  Better than out of the box Social communities for SharePoint.  They can integrate Yammer feeds into activity streams!
  • LimeLeap has a web-forms-on-SharePoint product called Pistachio
  • ShareGate – Migration tools
  • Actiance – compliance solutions for social computing
  • BA Insight is now focused on helping you create search-based applications (whereas before I thought of them primarily for their Longitude product which enhances the SharePoint search experience with rich previews).
  • Ontolica by Surfray was also there. 
  • Both BA Insight and Surfray offer ways of enhancing the SharePoint out of the box search experience with better previews and more. See http://www.surfray.com/products and http://www.bainsight.com/Pages/sharepoint-fast-search-document-preview.aspx
  • Axceler – Management tools for SharePoint
  • Idera – SharePoint tools for administration and bulk metadata updates,  tools for PowerShell, SQL, and backup.
  • mylyli Arc – a solution for publishing intranet content to an extranet without content duplication.
  • SharePoint Hosting – Rackspace and FPWeb were there.  FPWeb hosts my blog and CSPUG.org.  I’ve also had a client that has used Rackspace successfully, so I only have nice things to say about both companies.
  • A number of companies have a variety of different solutions for compliance and data security. Among these are:
    • Imperva.  Again over simplifying, they provide a set of application firewalls, such as a web app firewall, a db firewall, and a file server firewall.
    • Varonis – Admin tools focused on “data governance”
    • Both of these offer, among other things, a way to report on security / permission assignments.
  • CLM Matrix has a solution focused on management of contract documents.
  • Arx CoSign provides a way of applying cryptography based (rather than digital ink based) signatures to documents in a SharePoint site.  No more printing out documents just so they can be signed and scanned back in!
  • HP was there demonstrating a scanner that scans documents directly into document libraries with the help of an application that watches a network share for the scanned files.

Although they weren’t in the exhibit hall, I met with several people from Claysys and got a brief product demo of their InfoPath-like web-based-forms-and-workflow-on-top-of-SharePoint product.  Makes it even easier to build business applications via configuration rather than coding….

I didn’t include links to websites for each product above, because I want to get this post written tonight and you can Bing on any of the product or company names and I am sure you will find them.

On the second day, I had the honor of co-presenting a PowerShell workshop with Todd Klindt.  This was an open format session where attendees could bring their SharePointy PowerShell questions, and we would try to answer them live.

My slide deck for this workshop is at PowerShell for SharePoint Workshop although I also showed some slides from PowerShell for SharePoint Developers and Administrators and there is significant overlap between the two. 

One of the questions asked was about checked out files with no checked in versions, and I pointed out the link on the list settings page to view these.  That’s ManageCheckedOutFiles.aspx in _layouts. 

[Sidenote: Handy tip for being able to view POSH scripts in Windows Explorer’s Preview Pane]

To further explore the issue, I uploaded a file to a document library, and then we looked at some of its properties, in particular, the Level property.

$web =get-spweb $siteURL
$list = $web.Lists["Shared Documents"]
$doc = $list.items[0]

As I explained at the time, indexing into the items collection is fine when you know you have only a few (in this case one) list items.  However, it can have significant memory impact when it’s a large list.

Once we had the document in $doc, we could do things like this:

$doc.File.Level

#This showed that the file was checked out.

$doc.File.CheckIn("checked in by code")

$doc.File.CheckOut()

$list.Items | foreach {write-host $_.Name, $($_.File.Level)}

#could be memory intense for a large list

 

Todd talked about how the SharePoint Product Team had to comply with the naming standards set forth by the PowerShell product team.  Here’s how to see all the verbs that are used in SharePoint cmdlets:

get-command -noun "SP*" |foreach {$_.Name.Split("-")[0]} | select -Unique

Someone asked a question about running these cmdlets remotely.  Since we are working on the server object model, they need to execute on the Web Front End or an App Server.  However, you can use a remote PowerShell session. To find out more about running PowerShell commands remotely on a server, get-help about_Remote . To find out about the commands you can use to work with remote sessions, try: get-command -name "*pssession" There was another question about the User Information List, which caused me to do this:

$web = get-spweb http://pac.contoso.local/sites/bus

$web.Lists

$web.Lists | select -p Title

$web.Lists["User Information List"]

$web.Lists["User Information List"].items | Out-GridView

Note that Out-GridView only exists if you have the ISE installed.  Todd talked about how he doesn抰 use the ISE because you can抰 use Start-Transcript in the ISE. Whereas I do most of my work in the ISE, and use the script pane rather than the execution pane so I can keep track of what I抳e done.

Finally, someone had a question about getting a list of the servers in a farm, so I did this:

Get-SPServer

get-help get-spserver -examples

Another small PowerShell example is that earlier that morning, I had saved my presentation deck to JPEGs, but the first 9 slides had single digit slide numbers.  Here抯 a little script to rename those slides to prefix the single digit with a zero:

Get-ChildItem "C:\Users\michaelbl\Documents\profdev\2012\09-Sept\SPFest\jpegs" | where Name -like "Slide?.JPG" | foreach {
$newname = "Slide0" + $_.Name.Substring(5)
Move-Item $_.Name $newname
}

Had no one asked questions, I was prepared with a few scripts to show off.   Since we had a good set of questions, I never had a chance to show this:

  1. First, I started with a blog post of Todd抯: http://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=362
  2. I ignored the download link he provided, and copied the script into the ISE, INCLUDING THE LINE NUMBERS, then saved it as a file called CreateUsersTK.ps1
  3. To clean up the line numbers, I just removed the first three characters from each line:

get-content .\createUsersTK.ps1 | foreach {
$_.Substring(3)
} > .\createUsersTK2.ps1

After that, I took the script and giftwrapped it by turning it into a function which you can find here:giftwrap.ps1.  Note that I've not tested the result yet.

We gave a away a few books at the end of the session. After that, I attended a few sessions and spent more time in the exhibit hall, and had a chance to sit down with Jamie Story and Leo Doyle to plan upcoming CSPUG meetings.

Alisa Swann gave a good presentation on what's new in Office 2013.  Seamlessly switching between devices and your recent file list syncs  - cool.

I think that about covers it.

I'm looking forward to the next one!

--Michael


# Comments:
Modified: 10/8/2012 8:58 PM
Permalink: http://blumenthalit.net/blog/Lists/Posts/Post.aspx?ID=170
September 2012 in Chicago–Full of SharePoint Events!

 

There’s a lot going on next month that is SharePoint related! Check out these events:

  1. The Elgin  SharePoint Users Group meets on the 5th.
  2. SharePoint Fest is coming to Chicago (Rosemont really) September 25-27. 
    1. There are some excellent preconference sessions (including Dan Holme’s governance workshop, not to be missed!).
    2. That evening, the Chicago SharePoint Users Group (which I help run), will have a panel discussion featuring some of the speakers from the conference.  The following day, I’ll be giving my presentation on PowerShell and SharePoint.  Again, PowerShell is a key tool for working with SharePoint (2007,2010, and 2013), both for developers and for Admins, and if you don’t know the PowerShell language, now’s the perfect time to start! 
    3. On the last day of the conference, I have the honor of co-presenting a PowerShell Workshop with SharePoint MVP and expert SharePoint Admin Todd Klindt!  Bring all your PowerShell questions!
    4. The conference website is http://www.sharepointfest.com/Chicago/.  Use coupon code CSPUG-15 to save 15% when you register today!
  3. SharePoint Saturday Chicago is slated for September 29th.  Watch http://sharepointsaturday.org/chicago/default.aspx for more info.

If you are are interested in Business Intelligence, you should be aware that Magenic is having a Business Intelligence Summit in Chicago August 23rd.  More info at: http://magenic.com/Locations/ChicagoMidwestRegion/BISummit.aspx

If you are a SharePoint Developer, but SharePoint isn’t all you do, then this event may also be of interest: Code Mastery Chicago. It’s on September 12th, at Microsoft’s Downers Grove office.

--Michael

SPFestChicago2012


# Comments:
Modified: 8/14/2012 12:47 PM
Permalink: http://blumenthalit.net/blog/Lists/Posts/Post.aspx?ID=169
 
 
I also hold an MCSE, MCAD, MCDBA, and CAPM.
 
 

 INETA Champion Q3 2010, Q2 2011, Q2 2012

 

INETA Community Champions

 

 Recent Noteable Activities

 
  • Ramping up on SharePoint 2013
  • Planned and executed several MOSS 2007 to SharePoint 2010 migrations
  • Developed a custom document management site template solution
  • Served as a SharePoint Architect and Build Engineer on a SharePoint Intranet project for a Fortune 50 company.
  • Performed SharePoint Administration of a customer's intranet for several months.
  • Built a collaboration solution for a global civil engineering firm's communities of practice.
  • Hiring more SharePoint professionals at Magenic. Are you a SharePoint Developer with SharePoint Administration experience or a SharePoint Designer? Send me your resumé!
  • Public speaking on SharePoint topics. For more information, click here.
  • Webmaster for the Chicago SharePoint User Group. Visit CSPUG.org and come to our meetings!

 

    follow me on Twitter
     

     MSDN and TechNet Articles I've Commented On

     
     HideCustomAction Element (Custom Action)
     How to Add Actions to the User Interface
     Solution Schema
     SPList SDK Object Model Refernce
     View Element (View)
     Welcome to the Windows SharePoint Services 3.0 SDK
     ContentType Element (ContentType)
     Configuring IntelliSense with CAML Files When Developing for WSS 3.0
     Features and solutions: Stsadm operations (Office SharePoint Server)
     PowerShell Pack for SharePoint
     SPWorkflow.CreateHistoryEvent Method
     http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spfieldurlvalue_properties.aspx
     Configure Kerberos authentication (Office SharePoint Server) (Updated: 2009-03-26)