pouët.net

While it is indeed cool, "Debris" is much more of an "amazing achievement" IMHO.

ps: quite probably once the images are fed to their algorythm the process is completely automatic, I don't know if it's still a fronteer regarding automatic photogrammetry (I've found some related math stuff here)

What amazes me is in fact this kind of automatized data extraction working over sets of thousands of hi-res photos, being able to reconstruct a 3D (textured) geometry.

Then the scene rendering should be the most traditional task.

( ok, still no realtime bbs :] )
same as GbND.

That thing reaks of precalc btw. One-time operation to process all the images, and then you manouver around in some reduced datasets that streams more data when it needs to.

The comparative algorithm is cool though.

gb: you have any idea how hard it is to acquire and match markers on pictures? i cant seriously believe they have managed to make it an automatic process. even if it is automatic, the processing time it'll take to find all the matches and validate them would probably take longer then if someone would be selecting a couple markers on each pair of pictures by hand. i've done some researching on camera tracking and such, i'd be very pleasantly surprised if they'd found an automated process for matching all of these but i seriously doubt it.

From my previous post: "by relying on amateur material, it's going to be difficult to ensure reliability and accuracy of data" On second thoughts this is a problem of anything that uses stuff created by anyone and everyone rather than a problem of this technology.

I still think it's impressive and ground-breaking as I've never seen anything like it before. I don't mean I haven't seen anything implementing 3D or texture mapping as this is a lot more complicated than it looks from the outset. Can someone show me something else which is similar and has already been done? I'm asking this more out of interest rather than a challenge or anything.

broderick: 3d reconstruction out of several pictures has been around for a few years already, its the cross-identifying of the markers that'll allow for the 3d reconstruction thats usually a bitch. once you have a first 3d model it becomes easier to identify pictures belonging to the scene, but one would still need to get a few of markers matched to make it a fast search.. i'm very curious to read their papers about their engine.. have to look for them. but i bet there is some dirty hack bypassing the thing (i.e. someone in india matching markers on all pictures for them).

GbND + Broderick: Are you sure this is automatic? Surely each scene (being real life) is far too complex to calculate not to mention people and other stuff getting in the way. That maths example is plausible but the scene is relatively simplistic. I don't remember reading anywhere that SeaDragon (the name of the rendering engine) could do this but I might be wrong. There's the issue of colour and lighting too (e.g. different times of day, different cameras and settings etc.).

Is it even being textured on a model? Or is it more about calculating relative distances from the images? I should probably go read up more rather than posting this much. :/

LoL :DD

I suppose they might be using a combination of chromatic information and rectilinear features extraction such as windows, corners... and also stuff in them (as they say here), I remember having read something on the subject some time ago when willbe posted a link in the oneliner regarding a similar research...
I strongly doubt that the algorythm would work well with more natural-looking things like trees or cascades.

And yeah, feature matching is the black magic here... =O

ps: yes, I'm sure. I -work- here. I saw this, and asked these questions a year ago, before it was publicly released, and I saw it at techfest, our internal research fair. Their reponse to the data crunching time was that it was really, really long. The 'seadragon' viewer client doesn't do the (mostly geometry-based) feature-finding. It will show you the 'point cloud' of all of the recognised points. Yes, it works better on blocky geometry like buildings than trees. No, there's no minimum-wage india worker step. Workers in India don't scale. Server farms scale.

Read the press releases from Broderick's link. This isn't the only MS research project that uses feature extraction from arbitrary images. There are WAY crazier things going on, things that may or may not ever see the light of day. Trust me, if you don't trust my employer. I've seen things that make your questions sound like someone doubting that environment maps can be hardware accelerated. Call bullshit if you want to. Just remember, a year from now, who told you it was real.

Tell that to American post offices!

microsoft doing a demo = LOL

gb: hmmm cool then.

kewl

GbND: is this rather professional or mainstream-oriented ?

Zest: my guess is mainstream, to one-up Picasa, or to integrate with live. The other stuff I've seen is pure research that could go anywhere.

Agreed with broderick.

Basically most real world stuff can be split down in horizontals and verticals, and if you don't find horizontals and verticals after having done the edge/contrasts detection, then it's probably because there is perspective, and finding the inverse computations to "invert perspective' a picture is not something very complicated, even my 4 years old paintshop pro do it without any problem.

When the data on the image are "normalized" you can probably compute metrics, like the ratio between intersections and splits, the density of remarkable features, and build a dictionary. A bit like what they use for handwritten recognition and OCR.

By doing that on small chunks, taking into consideration neighbors overlap, you also solve the problem of creating the giant stream-able mosaic of picture elements.

Not easy, definitely a lot of work, but certainly doable :)