Skip to content

Commit

Permalink
update website
Browse files Browse the repository at this point in the history
  • Loading branch information
wolfstam committed Oct 23, 2024
1 parent f70ae95 commit ffed517
Show file tree
Hide file tree
Showing 10 changed files with 50 additions and 27 deletions.
77 changes: 50 additions & 27 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -174,20 +174,21 @@ <h3 class="subtitle has-text-centered">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
The challenge in object-based visual reasoning lies in generating descriptive yet distinct concept
representations. Moreover, doing this in an unsupervised fashion requires human users to understand a
model's learned concepts and potentially revise false concepts. In addressing this challenge, we
introduce the <span class="method">Neural Concept Binder</span>, a new framework for deriving discrete
concept representations
resulting in what we term “concept-slot encodings”. These encodings leverage both "soft binding" via
object-centric block-slot encodings and "hard binding" via retrieval-based inference. The <span
class="method">Neural Concept
Binder</span> facilitates straightforward concept inspection and direct integration of external
knowledge, such
as human input or insights from other AI models like GPT-4. Additionally, we demonstrate that
incorporating the hard binding mechanism does not compromise performance; instead, it enables seamless
integration into both neural and symbolic modules for intricate reasoning tasks, as evidenced by
evaluations on our newly introduced CLEVR-Sudoku dataset.
The challenge in object-based visual reasoning lies in generating concept representations
that are both descriptive and distinct. Achieving this in an unsupervised manner requires
human users to understand the model's learned concepts and, if necessary, revise incorrect
ones. To address this challenge, we introduce the <span class="method">Neural Concept Binder</span> (NCB),
a novel framework for deriving both discrete and continuous concept representations, which
we refer to as “concept-slot encodings”. NCB employs two types of binding: “soft binding”,
which leverages the recent SysBinder mechanism to obtain object-factor encodings, and subsequent
“hard binding”, achieved through hierarchical clustering and retrieval-based inference.
This enables obtaining expressive, discrete representations from unlabeled images.
Moreover, the structured nature of NCB's concept representations allows for intuitive
inspection and the straightforward integration of external knowledge, such as human input
or insights from other AI models like GPT-4. Additionally, we demonstrate that incorporating
the hard binding mechanism preserves model performance while enabling seamless integration into
both neural and symbolic modules for complex reasoning tasks. We validate the effectiveness of
NCB through evaluations on our newly introduced CLEVR-Sudoku dataset.
</p>
</div>
</div>
Expand All @@ -200,11 +201,12 @@ <h2 class="title is-3">Abstract</h2>


<section class="section">

<div class="container is-max-desktop">

<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Learn Inspectable and Revisable Concepts</h2>
<h2 class="title is-3">Learn Expressive, yet Inspectable and Revisable Concepts</h2>

<div class="content has-text-justified">
<p>
Expand All @@ -222,6 +224,17 @@ <h2 class="title is-3">Learn Inspectable and Revisable Concepts</h2>
</div>
<br />

<!-- Video Embed Section -->
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">NCB Step by Step</h2>
<video style="width: 100%; max-width: 1000px; height: auto;" controls>
<source src="./static/images/Presentation_website.mp4" type="video/mp4">
<p>Your browser does not support the video element. <a href="./static/images/Presentation_website.mp4">Download the video</a>.</p> </video>
</div>
</div>
<!--/ Video Embed Section -->

<!-- Interpretability. -->
<div class="columns is-centered">
<div class="column is-full-width">
Expand Down Expand Up @@ -263,9 +276,22 @@ <h2 class="title is-3">CLEVR Sudoku</h2>
varying degrees of difficulty. Each image is annotated with the correct solution to the puzzle, which
serves as the ground truth for evaluating the model's performance.
</p>
<p>CLEVR-Sudoku requires visual perception as well as deductive reasoning skills. We as humans are able to
abstract the relevant information from the images to reason about missing cells. Try for yourself!</p>

<p><strong>
CLEVR-Sudoku requires visual perception as well as deductive reasoning skills. Now you can try solving the puzzles yourself directly on this website!
Simply place the images where you think they belong and see if its right!
</strong></p>

<!-- Call to Action Button -->
<div class="has-text-centered">
<a href="#sudoku-container" class="button is-primary is-large is-rounded">
<span class="icon">
<i class="fas fa-play-circle"></i>
</span>
<span>
Play CLEVR-Sudoku Now!
</span>
</a>
</div>
</div>

<!-- Sudoku -->
Expand Down Expand Up @@ -338,15 +364,12 @@ <h2 class="title">BibTeX</h2>
<div class="column is-8">
<div class="content">
<p>
This work was supported by the Priority Program (SPP) 2422 in the subproject "Optimization of active
surface
design of high-speed progressive tools using machine and deep learning algorithms" funded by the
German Research Foundation (DFG), the "ML2MT" project from the Volkswagen Stiftung and the
"The Adaptive Mind" project from the Hessian Ministry of Science and Arts (HMWK). It has further
benefited from the HMWK projects "The Third Wave of Artificial Intelligence - 3AI", and Hessian.AI,
as well as the Hessian research priority program LOEWE within the project WhiteBox, the ICT-48
Network of AI Research Excellence Center "TAILOR" (EU Horizon 2020, GA No 952215) and the
EU-funded "TANGO" project (EU Horizon 2023, GA No 57100431).
This work was supported by the Priority Program (SPP) 2422 in the subproject “Optimization of active surface design
of high-speed progressive tools using machine and deep learning algorithms“ funded by the German Research Foundation (DFG),
the ”ML2MT” project from the Volkswagen Stiftung and the ”The Adaptive Mind” project from the Hessian Ministry of Science
and Arts (HMWK). It has further benefited from the HMWK projects ”The Third Wave of Artificial Intelligence - 3AI”,
and Hessian.AI, as well as the Hessian research priority program LOEWE within the project WhiteBox, and the EU-funded
“TANGO” project (EU Horizon 2023, GA No 57100431).
</p>
<p>The website template is based on the source code of <a
href="https://github.com/nerfies/nerfies.github.io">this
Expand Down
Binary file added docs/static/images/Presentation_website.mp4
Binary file not shown.
Binary file added docs/static/images/_motivation.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/static/images/counting.png
Binary file not shown.
Binary file removed docs/static/images/generalization.png
Binary file not shown.
Binary file modified docs/static/images/inspection.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/static/images/library.png
Binary file not shown.
Binary file modified docs/static/images/main.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/static/images/motivation.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/static/images/motivation.pdf
Binary file not shown.

0 comments on commit ffed517

Please sign in to comment.