{"id":213,"date":"2020-06-16T15:29:26","date_gmt":"2020-06-16T15:29:26","guid":{"rendered":"https:\/\/realitylab.uw.edu\/staging\/?p=213"},"modified":"2020-06-17T19:49:06","modified_gmt":"2020-06-17T19:49:06","slug":"background-matting-at-cvpr20-and-build-conference","status":"publish","type":"post","link":"https:\/\/realitylab.uw.edu\/staging\/?p=213","title":{"rendered":"Background Matting at CVPR&#8217;20 and Build Conference"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Our UW Reality Lab project on background matting has been getting attention lately &#8211; at Microsoft\u2019s Build Conference, and in this <a href=\"https:\/\/youtu.be\/6ml8kr1M1Pw?t=509\">Fireside Chat with Microsoft CEO Satya Nadella<\/a>:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>\u201cThe next area, Harry, that we\u2019re all so excited about in this world of remote everything, is Background Matting &#8230;we recently had the Build Developer Conference and we were able to take  presenters who just recorded themselves at home, and then we were able in fact to superimpose them in a virtual stage without needing that Green Screen. &#8230;Again, breakthroughs in computer vision that, in fact, we worked with the University of Washington on. So let\u2019s roll the video.\u201d <\/p><cite>&#8211; Satya Nadella<\/cite><\/blockquote>\n\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"CVPR 2020 Keynote: Satya Nadella\" width=\"800\" height=\"450\" src=\"https:\/\/www.youtube.com\/embed\/vgdVIeQKH-E?start=567&#038;feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">David Carmona <a href=\"https:\/\/blogs.microsoft.com\/ai-for-business\/creating-a-virtual-stage\/\">describes<\/a> just how important this project was to making the 2020 <em>Build Conference<\/em> happen during a global pandemic:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>&#8220;Covid-19 has made it impossible for large scale in-person events, so we  are all looking for new ways to communicate with our customers. And  again, if you participated in any part of Microsoft Build, then you experienced first-hand how our Microsoft Global Event\u2019s team had to  shift everything to virtual.&#8221;<br>&#8220;&#8230;So, we built on top of the background matting process by using our Azure Kinect sensors with an AI model based on the work from University of Washington to create a new way for our presenters to easily record  themselves from their home and appear on our virtual stage.&#8221;<\/p><cite>&#8211; David Carmona<\/cite><\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">Here&#8217;re the project page and some of its results:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/grail.cs.washington.edu\/projects\/background-matting\/\"><strong>Background Matting: The World Is Your Green Screen<\/strong><\/a><br><em><strong>(Authors:<\/strong><\/em> <a href=\"https:\/\/homes.cs.washington.edu\/~soumya91\/\">Soumyadip Sengupta<\/a>, <a href=\"https:\/\/www.vivekjayaram.com\/research\">Vivek Jayaram<\/a>, <a href=\"https:\/\/homes.cs.washington.edu\/~curless\/\">Brian Curless<\/a>, <a href=\"https:\/\/homes.cs.washington.edu\/~seitz\/\">Steve Seitz<\/a>, <a href=\"https:\/\/sites.google.com\/view\/irakemelmacher\/\/\">Ira Kemelmacher-Shlizerman<\/a>)<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Background Matting: Short Teaser\" width=\"800\" height=\"450\" src=\"https:\/\/www.youtube.com\/embed\/QCSYBt_G7C8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p><strong>Abstract: <\/strong>We propose a method for creating a matte \u2013 the per-pixel foreground color and alpha \u2013 of a person by taking photos or videos in an everyday setting with a handheld camera. Most existing matting methods require a green screen background or a manually created trimap to produce a good matte. Automatic, trimap-free methods are appearing, but are not of comparable quality. In our trimap free approach, we ask the user to take an additional photo of the background without the subject at the time of capture. This step requires a small amount of foresight but is far less timeconsuming than creating a trimap. We train a deep network with an adversarial loss to predict the matte. We first train a matting network with supervised loss on ground truth data with synthetic composites. To bridge the domain gap to real imagery with no labeling, we train another matting network guided by the first network and by a discriminator that judges the quality of composites. We demonstrate results on a wide variety of photos and videos and show significant improvement over the state of the art.<\/p><\/blockquote>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"199\" src=\"https:\/\/realitylab.uw.edu\/staging\/wp-content\/uploads\/2020\/06\/MattingTeaser-1024x199.png\" alt=\"\" class=\"wp-image-229\" srcset=\"https:\/\/realitylab.uw.edu\/staging\/wp-content\/uploads\/2020\/06\/MattingTeaser-1024x199.png 1024w, https:\/\/realitylab.uw.edu\/staging\/wp-content\/uploads\/2020\/06\/MattingTeaser-300x58.png 300w, https:\/\/realitylab.uw.edu\/staging\/wp-content\/uploads\/2020\/06\/MattingTeaser-768x149.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Our UW Reality Lab project on background matting has been getting attention lately &#8211; at Microsoft\u2019s Build Conference, and in this Fireside Chat with Microsoft CEO Satya Nadella: \u201cThe next&#8230; <a class=\"read-more-link\" href=\"https:\/\/realitylab.uw.edu\/staging\/?p=213\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":8,"featured_media":239,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[1],"tags":[27],"class_list":["post-213","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-matting","gt-excerpt","gt-excerpt-thumbnail-square"],"_links":{"self":[{"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/posts\/213","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=213"}],"version-history":[{"count":28,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/posts\/213\/revisions"}],"predecessor-version":[{"id":287,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/posts\/213\/revisions\/287"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=\/wp\/v2\/media\/239"}],"wp:attachment":[{"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=213"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=213"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/realitylab.uw.edu\/staging\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=213"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}