May 05

“Ask a Geek”

Posted in: CITRT, General IT, General Ministry

Part of our job in IT is to lead ministries in how to use technology.  Here is a quick email about an event that we are doing with our staff at Watermark next week.

Mark your calendars for noon next Thursday, May 14th for “Ask a Geek”.

Ever wonder what your friends are talking about when they mention Hulu?  Do you think Twitter is a gardening tool?  Have you wanted to blog but were afraid of being labeled as a blogger?  Want to be able to determine the links between SocialThing, CrazyEgg, Yoono, YouTube, Jiglu, Thoof, Zilo, OpenSocial, Facebook Connect and FriendFeed?  Confused?  You’re not alone.

Join us next Thursday at lunchtime for a brown bag session where you can ask ANY question about technology.  If you’re curious about how you can use some of these tools in your ministry, or just want to better understand the technology that surrounds us every day, this is your chance ask questions and explore.  We live in a time where technology is everywhere, so come learn how you might be able to leverage some of these tools for ministry.

Perhaps this is the most important thing that we can do for our ministries.  I often tell our team that we want to organize our technology so that we spend twenty percent of our time in maintenance mode and eighty percent of our time being technology evangelists.  We should always be innovators.

0 Comments
Apr 15

Adding Macs to an Active Directory Domain – Part One

Posted in: CITRT, General IT, Integration, Join the Journey, Mac OS X, Windows

We continue to add more and more Mac computers to our network.  Up until now, we have been happy to simply have the Macs be standalone machines, and haven’t wanted to undergo the learning curve to add them.  We are now at a point where managing user accounts and access has become difficult, and we want to move all of our Mac users onto network accounts.  Thus begins our journey into the abyss.

Starting out, the Mac integration seems simple enough.  Using the built-in Directory Utility, you are able to bind the Macs to the domain with very few issues.  Simply entering the fully qualified name of the active directory domain and clicking “Bind” should do the trick and add the machine to AD just like a Windows machine.

directory-utility-3

You are presented with several advanced options during this process.

directory-utility

Our first hurdle was determining what each of these options do.  In a nutshell, here were our findings (and our default settings above.)

  • Force local home directory:  This is something you will want to do, especially if you want applications to be available when users are off the network.  Not checking this box will place a ton of files on your server in the “network user home” directory.  That means that applications get installed here, the library folder is here, music is here, etc.  Not a good thing if you are a laptop user or are ever offline, and it creates more network traffic as well.
  • Use UNC path from AD:  Checking this box will map the AD user home folder upon login and will place it on the dock by default.  This is pretty useful if you have users who use their network folder a bunch.  In our situation, our network home folder in the windows environment was mapped to the user’s My Documents using Group Policy and offline files, so most of our users have a lot of documents there.
  • Default user shell:  Just leave it as it is.  /bin/bash works just fine.
  • Create mobile account at login:  This is the most tricky one. So we’ll talk about it later in detail.

Here is what show up under the mappings tab of Directory Utility.

directory-utility-1

Unless you really have a need to change anything here, it works great when left alone with the defaults.  And then we are left with the administrative tab.

directory-utility-2

The administrative tab is important for administering the machine.

  • Prefer this domain server: This is fine if you really want to set it, but is not really necessary in our case.
  • Allow administration by:  This is useful when the user is connected to the network and will allow the active directory network user account to make changes on the local machine… if they are a member of one of the groups added here.  In our testing, this seemed to only take effect at the GROUP level not at the user level, meaning we were unsuccessful in adding an individual user here for administration.  ALSO NOTE <ALERT> i.e. this is important! This seems to work fine when a user is connected to the network, however, what happens when the user is at home and can’t see the network?  Any guesses?  If you guessed that the user’s admin privileges are not cached and the user WILL NOT have administrative privileges when they are offline, you are correct.  Impact of this is that a network user goes home and while at their house needs to install an update, etc.  Because they can no longer see the network they will not be able to approve the update unless they have a second local user account with admin rights.  Not good, and probably confusing to the user.  This is a pretty major flaw in my opinion.

Mobile Accounts

Let’s talk a little more about creating mobile accounts. Creating the mobile account at login seems like a great thing to do.  In fact, it is the ONLY way that your users will be able to login using their network credentials when they are offline.  (Under Tiger, there was an option to cache credentials, which has now been replaced by Mobile Accounts.)  As we have previously mentioned though, if the user is offline they are no longer an admin, which really limits the ability of the user to administer the machine when off the network.  In addition, the mobile account user will be able to sync their folders between the network and the local machine.  This is great in theory, however not nearly as intuitive as Microsoft’s offline files, which automatically determined the latest version and save the appropriate one.  In the Mac implementation of sync, the user has to choose which version of the file to keep, which is dangerous in my opinion, and an extra burden on the user.  We choose NOT to sync the files for these reasons.

So where did we finally land on integrating our Macs with Active Directory?  Due to the limitations of offline administration and creating mobile accounts we decided to take a look at a few third party products.

Next time:  How we implemented Macs in Active Directory with Admitmac.

0 Comments | Tags: , , ,
Mar 30

Snapshots in VMware and how to survive them.

Posted in: CITRT, EqualLogic, General IT, VMware

vmwareWe survived a rather scary day with one of our main file servers on Sunday.  This server was one that we had only virtualized several weeks before and contains a large amount of critical data. I’m a huge fan of VMware and their products have transformed the way we do IT at Watermark, but yesterday was not fun.

Sunday morning I received a call that the server wasn’t responding, and on further review noticed that the server’s data store was completely out of space.  The server would start for a few minutes, but then error with “There is no more space in the redo log for servername-00002. You may be able to continue by freeing disk space on the relevant partition.”  This was the beginning of our lesson on VMware snapshots.  ***side note, we have Gold Support for VMware, which you think would be good, but no… if you want support outside the hours of 6am-6pm M-F, you need platinum support… Nice***  But I digress.

Last Saturday we had taken a snapshot which we had subsequently forgotten about.  When you take a snap in VMware, the system puts the original VMDK (virtual disk file) into a “holding pattern” and begins to write changes to a new virtual disk file, in our case the servername-00002 file.  The best practice of course, is to do a snapshot, make your changes, and then immediately delete the snapshot; at which point all of the changes will be written back into the original “holding pattern” VMDK and all is well.  Unfortunately, the system doesn’t do anything to remind you that the snap still exists if you forget to do this.  At the time of our discovery, the new 00002 file had grown to the size of 21 gigabytes and had filled up all of the available disk space.  This to me seems like something VMware should implement, a reminder that snaps are growing like crazy and about to take you out at the knees.

So our immediate course of action was to keep the server stopped (it wouldn’t run for more than a few minutes anyway before falling over), and get a complete copy of our file system from the SAN as a backup.  After copying nearly 60 gig from the SAN to a different location, it was time to attempt removal of the snapshot.

We went into Virtual Center, under Snapshots, and snapshot manager and saw the snapshot from last Saturday that we wanted to remove, and promptly removed it.  The task started and then hung at 95% for about fifteen minutes, at which we received a message that the “Operation Timed Out.”

Now, here is the kicker. You would think that a message like that would be a prompting to try again, but after lots of research it appears that the process has not really timed out at all.  Because the “tracking changes” VMDK is so large, it has lots of data to roll back into the original.  So in reality, the process is still running in the background and you just need to give it time to finish.  In fact, many people have said that reissuing the “remove snapshot” command will in-fact kill your data.  Not good VMware.

Fortunately, we found this information out before trying to remove the snap again.  Surely enough, two hours later, the process finished running and we were back to our original VMDK file.  The server started up with plenty of storage available again and all is well.

Like I said earlier, I absolutely love VMware, but I will not be using the snapshot capability in the future.  I think I will stick with EqualLogic snapshots, which seem to be faster AND safer.

I’d love to hear your comments on what we did right/wrong and how VMware has worked for you.

0 Comments | Tags: , , , ,
Mar 18

Should Memphis stick to BBQ and the Blues? Customer service fail.

Posted in: General IT, Shelby Arena

<begin rant> As most of you know, Watermark is an Arena and Shelby customer.  For the most part, we have been happy although today I got a response to a support call that I wasn’t to pleased about.  After doing a successful Shelby upgrade last Friday, I received a warning that our database hasn’t backed up for the past three days… Interesting.  All of our Arena backups have been fine, so this must be related to the upgrade.

After taking a look under the hood, I noticed that the schedule was still it place, it just wasn’t running.  When trying to recreate the schedule, sure enough, we get a Shelby application error and then the application quits.

I called Shelby support and finally talked to someone, but they had no idea what the issue was, so they went to grab someone in IT.

Then this happened… To give you at time reference, it is 4:45 pm, central time.  Memphis is in central time.

“Hey I was able to grab one of the guys from IT as he was headed out the door.  He has a pretty good idea what the problem is and thinks he can get you fixed up pretty quick.  IF YOU COULD JUST CALL US BACK FIRST THING IN THE MORNING, WE’LL GET YOU TAKEN CARE OF…”

Really?  I have a database that hasn’t backed up in 3 days and your IT guy can’t stop to help because he is on the way out the door at 4:45 in the afternoon?

Customer service fail.  I’m in love with the Arena product, but I’d be more in love with it if Shelby would learn to act like a hungry software company rather than a software company that mostly deals with churches.  </end rant>

0 Comments | Tags:
Sep 03

Good VMware help is hard to find

Posted in: General IT

Our VMware transition has been a bit of a headache lately.  Mainly because of all of the other things going on in tandum with trying to move to VMware ESX and our EqualLogic SAN.  To help us get over this hurdle, I asked our local EqualLogic/Dell rep to make a suggestion for a vendor who knows EQL and VMware very well.  What I was specifically looking for was a company who could help us implement best practices and make this solution scream.  What I got was something very, very different.

Now I’m not here to bag a company for just getting it wrong… but so far in the process I have found that I know more about how to implement VMware than they do.  Now that probably isn’t fair.  They are smart guys I’m sure, but with the help of all of the folks at #citrt (Church IT Roundtable IRC Chat) and others, we have a wealth of recent experience at using VMware and EqualLogic. Not just something we learned in training classes 5 releases ago.

Over the weekend, I found that some of the configurations they had recommended are not supported by VMware, and won’t work. (having 4 nics configured for iSCSI when it will only use 1, for you technically minded folks.)  When I pointed it out to them, they made recommendations on how it WOULD work.  Today they sent two people out to set it up their “recommended” way.  After they played for a few minutes on the system, even they realized that what they had “sold” as the solution, really won’t work.  One of the techs even went so far as to blame the sales guy!  So in the process of making the change over something that actually will, they managed to bring one of my ESX servers down and were unable to get it back online.  Now that means tomorrow I will be placing a call to VMware tech support to fix it myself.

I know that eventually we (Watermark) will get this working very, very well.  I have great confidence in what this solution will do for us, however I know now that good help may be harder to find than I would have thought.  If anyone has any options for local Dallas VMware solution provides, please let me know.

0 Comments
« Previous Page Next Page »