Colorado Copper Mountain Receiving Two New Lifts

[tball]

I made the mistake of looking at FB, but at least found they posted today that an update is coming:


I'm just hoping for the long term they double down on executing on the planned investments this summer and not back away from them.

 

[SBrown]

This is a fun feed ...

https://twitter.com/CopperCOMMS

That said, Breck had to evac a lift the other day, and today some family members were stuck twice for a combined hour plus and got so cold they had to quit early.

 

[fatbob]

I've been at Copper the past few days.

The only word to describe Centre Village is catastrophuck.

I never went down there but more or less every tourist I spoke to bitched about the 30+ min lines on Eagle. Today I never saw Eagle not stopped.

What madness compelled the idiots to try to replace their 2 most critical lifts at the same time then not bother starting until the fall? Yet they continue to lie about Flyer on TV commercials etc.

Even then sod all information telling people rocking up at the parking lots to avoid Centre Village. One might think they don't care about reputation.

 

[tball]

Looks like we've got an official message from Copper what's going on. They posted this exact language on Facebook a couple times yesterday afternoon:

What madness compelled the idiots to try to replace their 2 most critical lifts at the same time then not bother starting until the fall?

Add to that they are the first two lifts in North America using brand new direct drive technology.

The "electrical, software, hardware and mechanical systems" language is sounding like the disasters with the new technology for the DIA train crossing guards and the original DIA baggage system. In that context, it's nice to see the Eagle working at all.

With the benefit of hindsight, they should have done one lift this year and the other next year. I can also see how when they got the go-ahead from the corporate office, they just wanted to get them built. The money might not be there next year. These lifts are a big deal for Copper and will be around long after folks forget about this fiasco. I'm just hoping to still be around and still skiing when these lifts are eventually replaced.

 

[UGASkiDawg]

Looks like we've got an official message from Copper what's going on. They posted this exact language on Facebook a couple times yesterday afternoon:

View attachment 61939

Add to that they are the first two lifts in North America using brand new direct drive technology.

The "electrical, software, hardware and mechanical systems" language is sounding like the disasters with the new technology for the DIA train crossing guards and the original DIA baggage system. In that context, it's nice to see the Eagle working at all.

With the benefit of hindsight, they should have done one lift this year and the other next year. I can also see how when they got the go-ahead from the corporate office, they just wanted to get them built. The money might not be there next year. These lifts are a big deal for Copper and will be around long after folks forget about this fiasco. I'm just hoping to still be around and still skiing when these lifts are eventually replaced.

I'm not particularly upset at the problems. It's the piss poor communication that killing them. First hire for Dustin has to be a new head of communications. Getting the lifts running is LP's job and they'll get it done sooner or later. Communicating the what's, why's everyday is Coppers job.

 

[tball]

I'm not particularly upset at the problems. It's the piss poor communication that killing them. First hire for Dustin has to be a new head of communications. Getting the lifts running is LP's job and they'll get it done sooner or later. Communicating the what's, why's everyday is Coppers job.

Absolutely!

On that note, I've been watching this closely and this is the first I heard of their new Twitter feed with lift status (thanks @SBrown !):

This is a fun feed ...

https://twitter.com/CopperCOMMS

That said, Breck had to evac a lift the other day, and today some family members were stuck twice for a combined hour plus and got so cold they had to quit early.

I wonder what their communication strategy is for communicating this new communication strategy?

 

[tball]

Being a software engineer, that freaks me out. I think I'll avoid these lifts as much as is practical for a bit.

 

[dean_spirito]

Have they made any attempt to communicate what the issues are with American Eagle? Since the lift has become operational, it has broken down many times. It interfered with the Grand Prix and Revolution Tour events in the halfpipe. During both events, athletes were left stranded on the chair for 30-45 minutes. The event coordinators in both cases relied on snowmobiles to get athletes to the top of the course. Yesterday, people were stranded on the chair for over 30 minutes. Copper was able to get everyone off the lift without an evacuation, but Eagle was then shutdown for an hour + while lift maintenance fixed the issue.

This is a brand new chair. They say the technology is "state of the art". What is the deal? It seems like every time Copper tries to institute new technology, it only serves to further disrupt the operation and slow down the flow of traffic. IMO, they still haven't worked out all the bugs with the RFID turnstiles they installed last season.

 

[tball]

@dean_spirito the same technical issues that are delaying the Flyer seem to be causing the reliability issues with the Eagle. Copper posted an identically worded message about the Eagle as they did the Flyer:

Copper was able to get everyone off the lift without an evacuation, but Eagle was then shutdown for an hour + while lift maintenance fixed the issue.

What worries me is what they are doing to fix these issues with the Eagle on a daily basis while they are under extraordinary pressure.

Are they tweaking the operational parameters within those that have been tested by the tramway board? Or, are they updating software on the fly which would be extremely scary?

Is there any expertise in software configuration management and quality control on the tramway board? There's a long history of companies like LP that primarily build hardware being really bad at building high-quality software, while their products are becoming increasingly dependant on the software that runs their machines.

Given lives are on the line, ski lift software really should be engineered and regulated like medical device and aircraft software. I have a sneaking suspicion it is not. @Lift Blog I'd love to see a blog post from you about how lift software is built an managed!

 

[dbostedo]

Given lives are on the line, ski lift software really should be engineered and regulated like medical device and aircraft software.

Odds are that a failure of lift software is considered an inconvenience, not a life-or-death situation. (I.e. failure causes a stoppage or inconsistent operation - not something that would directly harm someone.) Certainly not on par with software failure in a modern airliner. A lift is a drastically simpler device.

 

[tball]

Odds are that a failure of lift software is considered an inconvenience, not a life-or-death situation. (I.e. failure causes a stoppage or inconsistent operation - not something that would directly harm someone.) Certainly not on par with software failure in a modern airliner. A lift is a drastically simpler device.

I respectfully disagree, as I expect would the family of Kelly Huber, the mother who died at Granby Ranch because of a malfunction of their lift's electrical drive/control system in 2017:
https://liftblog.com/2017/01/20/leitner-poma-to-install-new-drive-on-quickdraw-at-granby-ranch/
https://gazette.com/news/death-due-to-malfunction-heightens-attention-on-colorado-s-ski/article_f95e0eaf-bf09-59f5-9ba5-c87df302b40d.htm

Edit: Yes, a lift is a drastically simpler device. There are also external elements and dynamics that make the safe operation of lifts extraordinarily complex, as the Granby Ranch event highlights.

 

[dbostedo]

^^^
Hmm... neither of those articles points to a software failure. Are there others that do? Does software (or firmware, or middleware, etc.) have control over the lift in a way that could, say, jerk the chair to cause a fall? I'm very ignorant on this, and I'm making some assumptions about the electronic drives, and fail-safes and mechanical safety devices.

 

[tball]

Hmm... neither of those articles points to a software failure. Are there others that do? Does software (or firmware, or middleware, etc.) have control over the lift in a way that could, say, jerk the chair to cause a fall?

I don't know specifics about how lift software is built, but increasingly control system software is doing what was previously done by electronics. It was a while back, but I've done ten years of real-time software development for systems that interacted with the real world.

The retrofit at Granby Ranch replaced control system electronics on an old lift that I'd be willing to bet is managed by software in modern lifts. It's just the way things are going. Software is doing more and more. While the equivalent system is likely the drive control system firmware rather than software built by LP, it's just doing what it's told. I'd be surprised if you couldn't tell the drive controller to do something that could kill somebody.

What worries me is the new technology, tired engineers, holidays, and extraordinary pressure everyone is under. Note the new technology in these lifts is the direct drive. Add to that a potentially immature software engineering process, as is all too common. All speculation but informed as I've been around the block a few times.

This situation has many elements of an environment where bad things happen. I hope they take the time to figure out what's going on with the Eagle before pushing forward on the Flyer. They have a couple of weeks of low crowds until MLK weekend. I hope they make sure the lifts are working safely so this fiasco doesn't turn into a disaster.

 

[tball]

I found and read through the Granby Ranch investigation. It's quite the education on lift control systems and their testing and licensing. Such a tragic and sad, and fortunately rare, accident.


The conclusion was the accident was caused by improper programming of the drive controller along with the operator quickly changing the speed of the lift. There were a number of other possible contributing factors, but those were primary.

Copper's statement about malfunctioning lift automation software could mean a lot of different things, including I'm assuming the programming of the drive controller parameters. Given the Eagle shutting down and opening again, it sure seems like they must be tuning it in some way. Tuning a ski lift sounds like an art. There can't be much experience tuning these new direct drive systems.

It doesn't sound like they need to re-test for licensing after fine-tuning a lift, rather the changes need to be signed off by a P.E. as was done at Granby Ranch. In that case, the tramway board licensed a lift that had a drive control system that was deemed unsafe and needing replacement in less than a year.

They only test to determine a lift is operating correctly and never really dig into the black box, particularly the software, though they did suggest some licensing changes in that regard. When they dug in after the Granby Ranch accident they saw the new control system they licensed was unsafely engineered.

Personally, I'll feel safer riding Copper's new lifts when they haven't made any changes to them for a few weeks.

 

[pchewn]

High tech lifts with software: No thanks. At Mt Hood Ski Bowl on New Year's eve the lift was occasionally stopping and the operator would have to climb up into the engine house to re-set something and get the lift going again. This happened on and off several times. Finally, the expert technician arrived by snowmobile with a sophisticated tool to fix the root cause of the problem. The limit-switch wand for the cable tension was frozen and bent out of shape. Using the tool (a long pole with a weed digger zip-tied to the end), he whacked the limit switch wand back into position and the lift ran continuously for the rest of the day.

 

[tball]

It's going to be increasingly difficult to avoid lifts run by software.

From the Granby Ranch report, it sounds like all reasonably modern lifts run with digital motor controllers that are programmed with tuning parameters. The 1999 Granby Ranch HSQ upgrade replaced an old digital drive controller with a new one. That replacement is where things went really wrong.

Much older lifts have analog motor controllers, but it sounds like those are being replaced by digital controllers as the lifts are upgraded.

These motors and digital controllers are not specific for ski lifts, rather they are used in many industrial applications. They program and tune them for each ski lift application. This is nicely explained in the Granby Ranch report.

There is a wealth of experience tuning the old gear driven lifts, and from the report, it sounds like the parameters are fairly common. I'm guessing these new direct drive lifts behave differently and need to be tuned differently putting everyone involved is on a learning curve.

I've done a few smaller motor control systems myself and I can't imagine dealing with the dynamics of one of these huge machines. It's a wonder that they are so reliable and safe. It's just going to take some time to get used to the new technology. Direct drive should be inherently simpler and more reliable, but anything new is a challenge. I just worry about them mucking with things when they are carrying passengers.

 

[tball]

The 1999 Granby Ranch HSQ upgrade replaced an old digital drive controller with a new one. That replacement is where things went really wrong.

Correction, that should say: "The 2016 upgrade of the 1999 Granby Ranch HSQ replaced an old digital drive controller with a new one. That replacement is where things went really wrong. "

 

[Lift Blog]

I got a tour of the Winter Park direct drive gondola today. It is super simple and quiet. While this technology is new for Leitner-Poma of America, it has been used for more than 15 years on approximately 100 Leitner and Poma lifts elsewhere in the world. The new aspect is integrating Austrian manufactured drives and motors with otherwise American made lifts running American software. Two of the three initial LPOA direct drive lifts are up and running and the Flyer will be the third.

https://liftblog.com/the-gondola-winter-park-co/

 

[tball]

They were load testing the Flyer today.

We skied under the Flyer last run and the testing was quite the sight to see. They were doing some sort of loaded emergency stop test. After the forward motion stopped, the lift rolled back quite a way before finally stopping. The whole line then rocked and rolled for a while before it settled down. The rollback was maybe a third of the distance between the carriers. I have a video of my daughter yelling "Daddy It's going backward!!" They did the same test several times while we were skiing near it. Great to see it finally stopped each time!

I also heard a rumor they were bringing in some EE's to do additional programming or possibly did recently. Fingers crossed for the Flyer's behavior today being within specs so they can get it open shortly. Here's hoping it's not like the DIA train crossing gates that they still can't get to come down at the right time after two years and tens of millions of dollars.

I've never spent as much time in the Kokomo line as we did today, but it was still a super fun day and tolerable. We avoided Union Creek until the end of the day, and never went to Timberline assuming it wasn't worth it. That said, Copper, LP, and the Tramway Board, please take all the time you need to get the Flyer working safely. I'm pretty sure we were still better off than waiting in lines at Vail today.

 

[tball]

And, I got the answer to the important question that has been bugging me for weeks: what do they do with all the water in those boxes when the testing is done?

That was the one question I asked the large group of lift maintenance folks standing around during the testing. I had a zillion questions, of course, but that was the one I figure they could answer.

The first answer was "well, you are supposed to do this stuff in the summer" with a smile.

Turns out they are emptying the water at the top. They haven't built the exit snow ramp yet so they said they are making a huge ice monster there. When they are done load testing they will blow snow to cover it up and complete the ramp.

There is a hole on the bottom of each box and they poke a hole in the bag to drain the water from the box. What a messy and time-consuming operation. Much thanks to the crew for all their hard work to get these lifts going in difficult conditions. At least it was a warm day Saturday for that round of testing.



I haven't seen any updates on the status. It will be a week tomorrow since the last one, so time for another update, please!