I think for two years we were two months, uh, away from being done.

And what was that, Morgan? What project?

Uh, the, uh, TORRENT chip.

Oh.

Yeah. We were two we were.

Yeah.

Uh, uh, we went through it Jim and I went through old emails at one point and and for two years there was this thing saying, yeah, we're we're two months away from being done. It was very very believable schedules, too. I mean, we went through and with the schedules and we.

It was true for two years.

Yeah. Oh, yeah. It was very true.

So, should we just do the same kind of deal where we go around and do, uh, status report kind of things? OK. And I guess when Sunil gets here he can do his last or something. So.

Yeah. So we probably should wait for him to come before we do his.

Mm - hmm.

OK. That's a good idea.

Yeah.

OK.

Yeah.

Any objection? Do y OK, M

All in favor

Do you want to start, Morgan? Do you have anything, or?

Uh, I don't do anything. I No, I mean, I I'm involved in discussions with with people about what they're doing, but I think they're since they're here, they can talk about it themselves.

OK. So should I go so that, uh,

Yeah. Why don't you go ahead, Barry?

you're gonna talk about Aurora stuff, per se?

OK.

OK. Um. Well, this past week I've just been, uh, getting down and dirty into writing my my proposal. So, um Mmm. I just finished a section on, uh on talking about these intermediate categories that I want to classify, um, as a as a middle step. And, um, I hope to hope to get this, um a full rough draft done by, uh, Monday so I can give it to Morgan.

When is your, uh, meeting?

Um, my meeting

Yeah.

with, uh? Oh, oh, you mean the the quals.

The quals. Yeah.

Uh, the quals are happening in July twenty - fifth.

Oh. Soon.

Yeah.

Uh - huh.

D - Day.

Yeah.

Uh - huh.

So, is the idea you're going to do this paper and then you pass it out to everybody ahead of time and?

Right, right. So, y you write up a proposal, and give it to people ahead of time, and you have a short presentation. And, um, and then, um then everybody asks you questions.

Hmm.

Yeah.

I remember now.

Yep. So, um.

Have you d? I was just gonna ask, do you want to say any a little bit about it,

Y s

or? Mmm.

Oh. Uh, a little bit about?

Wh - what you're what you're gonna You said you were talking about the, uh, particular features that you were looking at,

Oh, the the.

or.

Right. Well, I was, um, I think one of the perplexing problems is, um, for a while I was thinking that I had to come up with a complete set of intermediate features in intermediate categories to to classify right away. But what I'm thinking now is, I would start with with a reasonable set. Something something like, um, um like, uh, re regular phonetic features, just to just to start off that way. And do some phone recognition. Um, build a system that, uh, classifies these, um these feat uh, these intermediate categories using, uh, multi - band techniques. Combine them and do phon phoneme recognition. Look at then I would look at the errors produced in the phoneme recognition and say, OK, well, I could probably reduce the errors if I included this extra feature or this extra intermediate category. That would that would reduce certain confusions over other confusions. And then and then reiterate. Um, build the intermediate classifiers. Uh, do phoneme recognition. Look at the errors. And then postulate new or remove, um, intermediate categories. And then do it again.

So you're gonna use TIMIT?

Um, for that for that part of the the process, yeah, I would use TIMIT.

Mm - hmm.

And, um, then after after, uh, um, doing TIMIT. Right?

Mm - hmm.

Um, that's that's, um that's just the ph the phone recognition task.

Yeah.

Uh, I wanted to take a look at, um, things that I could model within word. So, I would mov I would then shift the focus to, um, something like Schw - Switchboard, uh, where I'd I would be able to, um to model, um, intermediate categories that span across phonemes,

Mm - hmm.

not just within the phonemes, themselves, um, and then do the same process there, um, on on a large vocabulary task like Switchboard. Uh, and for that for that part I would I'd use the SRI recognizer since it's already set up for for Switchboard. And I'd run some some sort of tandem - style processing with, uh, my intermediate classifiers.

Oh. So that's why you were interested in getting your own features into the SRI files.

Yeah. That's why I I was asking about that.

Yeah. Yeah.

Yeah. Um, and I guess that's that's it. Any any questions?

Sounds good. So you just have a few more weeks, huh?

Um, yeah. A few more.

It's about a month from now?

It's a it's a month and and a week.

Yeah.

Yeah.

So, uh, you want to go next, Dave? And we'll do.

Oh. OK, sure. So, um, last week I finally got results from the SRI system about this mean subtraction approach. And, um, we we got an improvement, uh, in word error rate, training on the TI - digits data set and testing on Meeting Recorder digits of, um, six percent to four point five percent, um, on the n on the far - mike data using PZM F, but, um, the near - mike performance worsened, um, from one point two percent to two point four percent. And, um, wh why would that be, um, considering that we actually got an improvement in near - mike performance using HTK? And so, uh, with some input from, uh, Andreas, I have a theory in two parts. Um, first of all HTK sorry, SR - the SRI system is doing channel adaptation, and so HTK wasn't. Um, so this, um This mean subtraction approach will do a kind of channel normalization and so that might have given the HTK use of it a boost that wouldn't have been applied in the SRI case. And also, um, the Andreas pointed out the SRI system is using more parameters. It's got finer - grained acoustic models. So those finer - grained acoustic models could be more sensitive to the artifacts in the re - synthesized audio. Um. And me and Barry were listening to the re - synthesized audio and sometimes it seems like you get of a bit of an echo of speech in the background. And so that seems like it could be difficult for training, cuz you could have different phones lined up with a different foreground phone, um, depending on the timing of the echo. So, um, I'm gonna try training on a larger data set, and then, eh, the system will have seen more examples o of these artifacts and hopefully will be more robust to them. So I'm planning to use the Macrophone set of, um, read speech, and, um Hmm.

I had another thought just now, which is, uh, remember we were talking before about we were talking in our meeting about, uh, this stuff that some of the other stuff that Avendano did, where they were, um, getting rid of low - energy sections? Um, uh, if you if you did a high - pass filtering, as Hirsch did in late eighties to reduce some of the effects of reverberation, uh, uh, Avendano and Hermansky were arguing that, uh, perhaps one of the reasons for that working was ma may not have even been the filtering so much but the fact that when you filter a an all - positive power spectrum you get some negative values, and you gotta figure out what to do with them if you're gonna continue treating this as a power spectrum. So, what what Hirsch did was, uh, set them to zero set the negative values to zero. So if you imagine a a waveform that's all positive, which is the time trajectory of energy, um, and, uh, shifting it downwards, and then getting rid of the negative parts, that's essentially throwing away the low - energy things. And it's the low - energy parts of the speech where the reverberation is most audible. You know, you have the reverberation from higher - energy things showing up in So in this case you have some artificially imposed reverberation - like thing. I mean, you're getting rid of some of the other effects of reverberation, but because you have these non - causal windows, you're getting these funny things coming in, uh, at n And, um, what if you did? I mean, there's nothing to say that the the processing for this re - synthesis has to be restricted to trying to get it back to the original, according to some equation. I mean, you also could, uh, just try to make it nicer.

Uh - huh.

And one of the things you could do is, you could do some sort of VAD - like thing

Mm - hmm.

and you actually could take very low - energy sections and set them to some some, uh, very low or or near zero value. I mean, uh, I'm just saying if in fact it turns out that that these echoes that you're hearing are, uh.

Uh - huh.

or pre - echoes, whichever they are are are, uh, part of what's causing the problem, you actually could get rid of them.

Uh - huh.

Be pretty simple. I mean, you do it in a pretty conservative way

OK.

so that if you made a mistake you were more likely to keep in an echo than to throw out speech.

Hmm.

Um, what is the reverberation time like there?

In thi in this room? Uh.

On, uh, the the one what the s in the speech that you are you are using like?

Y Yeah. I I I I don't know.

So, it's this room.

It's, uh.

It's it's this room.

Oh, this room?

So.

OK.

so it's these are just microphone this micro close microphone and a distant microphone, he's doing these different tests on.

Oh.

Uh, we should do a measurement in here. I g think we never have. I think it's I would guess, uh, point seven, point eight seconds f uh, R T

Hmm!

something like that? But it's you know, it's this room.

Mm - hmm.

So.

OK. Mm - hmm.

Uh. But the other thing is, he's putting in w I was using the word " reverberation " in two ways. He's also putting in, uh, a he's taking out some reverberation, but he's putting in something, because he has averages over multiple windows stretching out to twelve seconds, which are then being subtracted from the speech. And since, you know, what you subtract, sometimes you'll be you'll be subtracting from some larger number and sometimes you won't. And.

Mm - hmm. Mm - hmm.

So you can end up with some components in it that are affected by things that are seconds away. Uh, and if it's a low energy compo portion, you might actually hear some funny things.

Yeah.

O o one thing, um, I noticed is that, um, the mean subtraction seems to make the PZM signals louder after they've been re - synthesized. So I was wondering, is it possible that one reason it helped with the Aurora baseline system is just as a kind of gain control? Cuz some of the PZM signals sound pretty quiet if you don't amplify them.

Mm - hmm. I don't see why why your signal is louder after processing, because yo

Yeah. I don't know why - y, uh, either.

Yeah.

I don't think just multiplying the signal by two would have any effect.

Mm - hmm.

Oh, OK.

Yeah. I mean, I think if you really have louder signals, what you mean is that you have better signal - to - noise ratio.

Well, well.

So if what you're doing is improving the signal - to - noise ratio, then it would be better.

Mm - hmm.

But just it being bigger if with the same signal - to - noise ratio.

It w i i it wouldn't affect things.

No.

Yeah.

OK.

Well, the system is use the absolute energy, so it's a little bit dependent on on the signal level. But, not so much, I guess.

Well, yeah. But it's trained and tested on the same thing.

Mmm.

So if the if the if you change in both training and test, the absolute level by a factor of two, it will n have no effect.

Mm - hmm. Yeah.

Did you add this data to the training set, for the Aurora? Or you just tested on this?

Uh Um. Did I w what?

Well, Morgan was just saying that, uh, as long as you do it in both training and testing, it shouldn't have any effect.

Sorry? Yeah.

But I I was sort of under the impression that you just tested with this data.

I I b

You didn't train it also.

I Right. I trained on clean TI - digits. I I did the mean subtraction on clean TI - digits. But I didn't I'm not sure if it made the clean ti TI - digits any louder.

Oh, I see.

I only remember noticing it made the, um, PZM signal louder.

OK. Well, I don't understand then. Yeah.

Huh. I don't know. If it's if it's like, if it's trying to find a a reverberation filter, it could be that this reverberation filter is making things quieter. And then if you take it out that taking it out makes things louder. I mean.

Uh, no. I mean, uh, there's there's nothing inherent about removing if you're really removing,

Nuh - huh.

uh, r uh, then I don't see how that would make it louder.

The mean. OK. Yeah, I see.

So it might be just some.

Yeah. OK. So I should maybe listen to that stuff again.

Yeah. It might just be some artifact of the processing that that, uh, if you're Uh, yeah. I don't know.

Oh. OK.

I wonder if there could be something like, uh for s for the PZM data,

Eh

uh, you know, if occasionally, uh, somebody hits the table or something, you could get a spike. Uh. I'm just wondering if there's something about the, um you know, doing the mean normalization where, uh, it it could cause you to have better signal - to - noise ratio. Um.

Well, you know, there is this. Wait a minute. It it i maybe i If, um Subtracting the the mean log spectrum is is is like dividing by the spectrum. So, depending what you divide by, if your if s your estimate is off and sometimes you're you're you're getting a small number, you could make it bigger.

Mm - hmm.

Mm - hmm.

So, it's it's just a a question of there's It it could be that there's some normalization that's missing, or something to make it.

Mm - hmm.

Uh, y you'd think it shouldn't be larger, but maybe in practice it is. That's something to think about.

Hmm.

I don't know.

I had a question about the system the SRI system. So, you trained it on TI - digits? But except this, it's exactly the same system as the one that was tested before and that was trained on Macrophone. Right? So on TI - digits it gives you one point two percent error rate and on Macrophone it's still O point eight. Uh, but is it exactly the same system?

Uh. I think so.

Hmm.

If you're talking about the Macrophone results that Andreas had about, um, a week and a half ago, I think it's the same system.

Mm - hmm. So you use VTL - uh, vocal tract length normalization and, um, like MLLR transformations also,

Mm - hmm.

and.

I'm sorry, was his point eight percent, er, a a result on testing on Macrophone or or training?

all that stuff.

That's.

It was training on Macrophone and testing yeah, on on meeting digits.

Oh. So that was done already. So we were Uh, and it's point eight? OK.

Mm - hmm.

OK.

Yeah. I I've just been text  testing the new Aurora front - end with well, Aurora system actually so front - end and HTK, um, acoustic models on the meeting digits and it's a little bit better than the previous system. We have I have two point seven percent error rate. And before with the system that was proposed, it's what? It was three point nine. So.

Oh, that's a lot better.

We are getting better.

So, what w?

And.

With the with the HTK back - end? What we have for Aurora?

Yeah. Two point seven.

I know in the meeting, like.

On the meeting we have two point seven.

Right. Oh.

That's with the new IIR filters?

Uh. Yeah, yeah. So, yeah,

OK.

we have the new LDA filters, and I think, maybe I didn't look, but one thing that makes a difference is this DC offset compensation. Uh, eh Do y did you have a look at at the meet uh, meeting digits, if they have a DC component, or?

I I didn't. No.

Oh.

Hmm.

No. The DC component could be negligible. I mean, if you are recording it through a mike. I mean, any all of the mikes have the DC removal some capacitor sitting right in that bias it.

Yeah. But this uh, uh, uh, no. Because, uh, there's a sample and hold in the A - toD. And these period these typically do have a DC offset.

Oh, OK.

And and they can be surprisingly large. It depends on the electronics.

Oh, so it is the digital OK. It's the A - toD that introduces the DC in.

Yeah. The microphone isn't gonna pass any DC.

Yeah. Yeah. Yeah.

But but,

OK.

typi you know, unless Actually, there are instrumentation mikes that that do pass go down to DC. But but,

Mm - hmm.

uh, no, it's the electronics. And they and.

Mm - hmm.

then there's amplification afterwards. And you can get, I think it was I think it was in the Wall Street Journal data that that I can't remember, one of the DARPA things. There was this big DC - DC offset

Mm - hmm.

we didn't we didn't know about for a while, while we were messing with it. And we were getting these terrible results. And then we were talking to somebody and they said, " Oh, yeah. Didn't you know? Everybody knows that. There's all this DC offset in th " So, yes. You can have DC offset in the data.

Oh, OK.

Yeah.

OK.

So was that was that everything, Dave?

Oh. And I also, um, did some experiments about normalizing the phase. Um. So I c I came up with a web page that people can take a look at. And, um, the interesting thing that I tried was, um, Adam and Morgan had this idea, um, since my original attempts to, um, take the mean of the phase spectra over time and normalize using that, by subtracting that off, didn't work. Um, so, well, that we thought that might be due to, um, problems with, um, the arithmetic of phases. They they add in this modulo two pi way and, um, there's reason to believe that that approach of taking the mean of the phase spectrum wasn't really mathematically correct. So, what I did instead is I took the mean of the FFT spectrum without taking the log or anything, and then I took the phase of that, and I subtracted that phase off to normalize. But that, um, didn't work either.

See, we have a different interpretation of this. He says it doesn't work. I said, I think it works magnificently, but just not for the task we intended. Uh, it gets rid of the speech.

What does it leave?

Uh, gets rid of the speech.

Uh, it leaves you know, it leaves the junk. I mean, I I think it's it's tremendous.

Oh, wow.

You see, all he has to do is go back and reverse what he did before, and he's really got something.

Well, could you take what was left over and then subtract that?

Ex - exactly. Yeah, you got it.

Yeah.

Yeah.

So, it's it's a general rule.

Oh, it's.

Just listen very carefully to what I say and do the opposite. Including what I just said.

And, yeah, that's everything.

All set? Do you want to go, Stephane?

Um. Yeah. Maybe, concerning these d still, these meeting digits. I'm more interested in trying to figure out what's still the difference between the SRI system and the Aurora system. And Um. Yeah. So, I think I will maybe train, like, gender - dependent models, because this is also one big difference between the two systems. Um, the other differences were the fact that maybe the acoustic models of the SRI are more SRI system are more complex. But, uh, Chuck, you did some experiments with this and

It didn't seem to help in the HTK system.

it was hard t to to have some exper some improvement with this. Um.

Well, it sounds like they also have he he's saying they have all these, uh, uh, different kinds of adaptation.

Mm - hmm.

You know, they have channel adaptation. They have speaker adaptation.

Yeah. Right.

Well, there's also the normalization.

Yeah. Yeah.

Yeah.

Yeah.

Like they do, um I'm not sure how they would do it when they're working with the digits,

The vocal tr

but, like, in the Switchboard data, there's, um conversation - side normalization for the non - C - zero components,

Yeah. Yeah. This is another difference. Their normalization works like on on the utterance levels.

Mm - hmm.

But we have to do it We have a system that does it on - line.

Right.

So, it might be it might be better with it might be worse if the channel is constant,

Yeah.

or Nnn.

And the acoustic models are like - k triphone models or or is it the whole word?

SRI it's it's tr

SRI.

Yeah.

Yeah. I guess it's triphones.

It's triphone.

I think it's probably more than that.

Huh.

I mean, so they they have I I thin think they use these, uh, uh, genone things. So there's there's these kind of, uh, uh, pooled models and and they can go out to all sorts of dependencies.

Oh. It's like the tied state.

So.

Mm - hmm.

They have tied states and I think I I I don't real I'm talk I'm just guessing here. But I think I think they they don't just have triphones.

OK.

I think they have a range of of, uh, dependencies.

Mm - hmm.

Mm - hmm.

Mm - hmm.

Hmm.

And Yeah. Well. Um. Well, the first thing I that I want to do is just maybe these gender things. Uh. And maybe see with Andreas if Well, I I don't know how much it helps, what's the model.

So so the n stuff on the numbers you got, the two point seven, is that using the same training data that the SRI system used and got one point two?

That's right. So it's the clean TI - digits training set.

So exact same training data?

Right.

OK.

Mm - hmm. I guess you used the clean training set.

Right.

Mm - hmm.

For with the SRI system.

Well.

You know, the the Aurora baseline is set up with these, um this version of the clean training set that's been filtered with this G - seven - one - two filter, and, um, to train the SRI system on digits S - Andreas used the original TI - digits, um, under U doctor - speech data TI - digits, which don't have this filter. But I don't think there's any other difference.

Mm - hmm. Mm - hmm. Yeah.

So is that? Uh, are are these results comparable? So you you were getting with the, uh, Aurora baseline something like two point four percent on clean TI - digits, when, uh, training the SRI system with clean TR digits  TI - digits. Right? And.

Um. Uh - huh.

Yeah. And, so, is your two point seven comparable, where you're, uh, uh, using, uh, the submitted system?

Yeah. I think so.

OK.

Yeah.

So it's about the same,

Mm - hmm.

maybe a little worse.

W w it was one one point two

Ye

with the SRI system,

I'm sorry.

Yeah.

I.

The complete SRI system is one point two.

You you were HTK.

Yeah.

Right? OK. That's right. So.

Mm - hmm.

OK, so the comparable number then, uh for what you were talking about then, since it was HTK, would be the um, two point f

It was four point something. Right? The HTK system with, uh, b

D d

Oh, right, right, right, right.

MFCC features.

Do you mean the b? The baseline Aurora - two system, trained on TI - digits, tested on Meeting Recorder near, I think we saw in it today, and it was about six point six percent.

Right. Right, right, right.

Oh.

OK. Alright. So He's doing some different things.

So Yeah. The only difference is the features, right now, between this and.

Yes. OK, good. So they are helping.

Mm - hmm.

That's good to hear. Yeah.

They are helping. Yeah. Um. Yeah. And another thing I I maybe would like to do is to just test the SRI system that's trained on Macrophone test it on, uh, the noisy TI - digits,

Yeah.

cuz I'm still wondering where this improvement comes from. When you train on Macrophone, it seems better on meeting digits. But I wonder if it's just because maybe Macrophone is acoustically closer to the meeting digits than than TI - digit is, which is TI - digits are very clean recorded digits

Mm - hmm.

and.

You know, it would also be interesting to see, uh to do the regular Aurora test,

Uh, f s

um, but use the SRI system instead of HTK.

That's Yeah. That's what I wanted, just, uh Yeah. So, just using the SRI system, test it on and test it on Aurora TI - digits. Right.

Why not the full Aurora, uh, test?

Um. Yeah. There is this problem of multilinguality yet.

Mm - hmm.

So we don't.

You'd have to train the SRI system with with all the different languages.

i i

Right.

We would have to train on.

Yeah. That's what I mean.

Yeah.

So, like, comple

It'd be a lot of work. That's the only thing.

Yeah.

Mmm.

It's.

Well, I mean,

Mmm.

uh, uh, I guess the work would be into getting the the files in the right formats, or something. Right? I mean.

Mm - hmm.

Because when you train up the Aurora system, you're, uh you're also training on all the data.

That's right.

I mean, it's.

Yeah. Yeah. I see. Oh, so, OK. Right. I see what you mean.

That's true, but I think that also when we've had these meetings week after week, oftentimes people have not done the full arrange of things

Mm - hmm.

because on on whatever it is they're trying, because it's a lot of work, even just with the HTK.

Mm - hmm.

So, it's it's a good idea, but it seems like it makes sense to do some pruning

Mm - hmm.

first with a a test or two that makes sense for you,

Yeah.

and then take the likely candidates and go further.

Yeah.

Mm - hmm. Yeah. But, just testing on TI - digits would already give us some information about what's going on. And mm - hmm. Uh, yeah. OK. Uh, the next thing is this this VAD problem that, um, um So, I'm just talking about the the curves that I I sent I sent you so, whi that shows that when the SNR decrease, uh, the current VAD approach doesn't drop much frames for some particular noises, uh, which might be then noises that are closer to speech, uh, acoustically.

I i Just to clarify something for me. I They were supp Supposedly, in the next evaluation, they're going to be supplying us with boundaries.

Mm - hmm.

So does any of this matter? I mean, other than our interest in it. Uh.

Uh Well. First of all, the boundaries might be, uh like we would have t two hundred milliseconds or before and after speech. Uh. So removing more than that might still make a difference in the results.

Do we? I mean, is there some reason that we think that's the case?

And No. Because we don't didn't looked that much at that.

Yeah.

But, still, I think it's an interesting problem.

Oh, yeah.

And Um. Yeah.

But maybe we'll get some insight on that when when, uh, the gang gets back from Crete. Because there's lots of interesting problems, of course.

Mm - hmm.

And then the thing is if if they really are going to have some means of giving us fairly tight, uh, boundaries, then that won't be so much the issue.

Yeah, yeah. Mm - hmm. Mm - hmm.

Um But I don't know.

Because w we were wondering whether that VAD is going to be, like, a realistic one or is it going to be some manual segmentation. And then, like, if if that VAD is going to be a realistic one, then we can actually use their markers to shift the point around, I mean, the way we want

Mm - hmm.

to find a I mean, rather than keeping the twenty frames, we can actually move the marker to a point which we find more suitable for us.

Right.

But if that is going to be something like a manual, uh, segmenter, then we can't use that information anymore,

Mm - hmm.

because that's not going to be the one that is used in the final evaluation.

Right.

So. We don't know what is the type of VAD which they're going to provide.

Yeah.

Yeah. And actually there's Yeah. There's an uh, I think it's still for even for the evaluation, uh, it might still be interesting to work on this because the boundaries apparently that they would provide is just, um, starting of speech and end of speech uh, at the utterance level. And Um.

With some some gap.

So.

I mean, with some pauses in the center, provided they meet that whatever the hang - over time which they are talking.

Yeah. But when you have like, uh, five or six frames, both.

Yeah. Then the they will just fill fill it up.

it it with.

I mean, th Yeah.

Yeah.

So if you could get at some of that, uh.

So.

although that'd be hard.

Yeah. It might be useful for, like, noise estimation, and a lot of other things that we want to work on.

But but Yeah.

Yeah.

Right. OK.

But Mmm. Yeah. So I did I just started to test putting together two VAD which was was not much work actually. Um, I im re - implemented a VAD that's very close to the, um, energy - based VAD that, uh, the other Aurora guys use. Um. So, which is just putting a threshold on the noise energy,

Mm - hmm.

and, detect detecting the first group of four frames that have a energy that's above this threshold, and, uh, from this point, uh, tagging the frames there as speech. So it removes the first silent portion portion of each utterance. And it really removes it, um, still o on the noises where our MLP VAD doesn't work a lot.

Mmm.

Uh,

Cuz I would have thought that having some kind of spectral information,

and.

uh uh, you know, in the old days people would use energy and zero crossings, for instance uh, would give you some better performance. Right? Cuz you might have low - energy fricatives or or, uh stop consonants, or something like that.

Mm - hmm.

Uh.

Yeah. So, your point is will be to u use whatever.

Oh, that if you d if you use purely energy and don't look at anything spectral, then you don't have a good way of distinguishing between low - energy speech components and nonspeech. And, um,

Mm - hmm.

just as a gross generalization, most nonsp many nonspeech noises have a low - pass kind of characteristic, some sort of slope. And and most, um, low - energy speech components that are unvoiced have a a high - pass kind of characteristic.

Mm - hmm.

an upward slope. So having some kind of a.

Yeah.

uh, you know, at the beginning of a of a of an S sound for instance, just starting in, it might be pretty low - energy,

Mm - hmm.

but it will tend to have this high - frequency component. Whereas, a a lot of rumble, and background noises, and so forth will be predominantly low - frequency. Uh, you know, by itself it's not enough to tell you, but it plus energy is sort of.

Yeah.

it plus energy plus timing information is sort of.

Mm - hmm.

I mean, if you look up in Rabiner and Schafer from like twenty - five years ago or something, that's sort of what they were using then.

Mm - hmm.

So it's it's not a.

Mm - hmm.

Hmm.

So, yeah. It it might be that what I did is so, removes like low, um, uh low - energy, uh, speech frames. Because the way I do it is I just I just combine the two decisions so, the one from the MLP and the one from the energy - based with the with the and operator. So, I only keep the frames where the two agree that it's speech. So if the energy - based dropped dropped low - energy speech, mmm, they they are they are lost. Mmm.

Mm - hmm.

But s still, the way it's done right now it it helps on on the noises where it seems to help on the noises where our VAD was not very good.

Well, I guess I mean, one could imagine combining them in different ways. But but, I guess what you're saying is that the the MLP - based one has the spectral information. So.

Yeah. But Yeah. But the way it's combined wi is maybe done Well, yeah.

Well, you can imagine.

The way I use a an a " AND " operator is So, it I, uh.

Is?

The frames that are dropped by the energy - based system are are, uh, dropped, even if the, um, MLP decides to keep them.

Right. Right. And that might not be optimal,

But, yeah.

but.

Mm - hmm.

No

but I mean, I guess in principle what you'd want to do is have a uh, a probability estimated by each one and and put them together.

Yeah. Mmm. M Yeah.

Something that that I've used in the past is, um when just looking at the energy, is to look at the derivative. And you make your decision when the derivative is increasing for so many frames. Then you say that's beginning of speech.

Uh - huh.

But, I'm I'm trying to remember if that requires that you keep some amount of speech in a buffer. I guess it depends on how you do it. But I mean, that's that's been a useful thing.

Yeah.

Mm - hmm.

Mm - hmm.

Yeah. Well, every everywhere has a delay associated with it. I mean, you still have to k always keep a buffer,

Mm - hmm.

then only make a decision because you still need to smooth the decision further.

Right. Right.

So that's always there.

Yeah. OK.

Well, actually if I don't maybe don't want to work too much of on it right now. I just wanted to to see if it's what I observed was the re was caused by this this VAD problem.

Mm - hmm.

And it seems to be the case. Um. Uh, the second thing is the this spectral subtraction. Um. Um, which I've just started yesterday to launch a bunch of, uh, twenty - five experiments, uh, with different, uh, values for the parameters that are used. So, it's the Makhoul - type spectral subtraction which use an over - estimation factor. So, we substr I subtract more, um, noise than the noise spectra that is estimated on the noise portion of the s uh, the utterances. So I tried several, uh, over - estimation factors. And after subtraction, I also add a constant noise, and I also try different, uh, noise, uh, values and we'll see what happen.

Hmm. OK.

Mm - hmm. Mm - hmm. But st still when we look at the, um Well, it depends on the parameters that you use, but for moderate over - estimation factors and moderate noise level that you add, you st have a lot of musical noise. Um. On the other hand, when you subtract more and when you add more noise, you get rid of this musical noise but maybe you distort a lot of speech. So. Well. Mmm. Well, it until now, it doesn't seem to help. But We'll see. So the next thing, maybe I what I will try to to do is just to try to smooth mmm, the, um to smooth the d the result of the subtraction, to get rid of the musical noise, using some kind of filter, or.

Can smooth the SNR estimate, also.

Yeah. Right. Mmm.

Your filter is a function of SNR. Hmm?

Yeah. So, to get something that's would be closer to what you tried to do with Wiener filtering.

Yeah.

And Mm - hmm. Yeah.

Actually, it's, uh Uh. I don't know, it's go ahead.

It.

And it's.

Maybe you can.

go ahead.

I think it's That's it for me.

OK. So, uh u th I've been playing with this Wiener filter, like. And there are there were some bugs in the program, so I was p initially trying to clear them up. Because one of the bug was I was assuming that always the VAD uh, the initial frames were silence. It always started in the silence state, but it wasn't for some utterances. So the it wasn't estimating the noise initially, and then it never estimated, because I assumed that it was always silence.

Mm - hmm. So this is on SpeechDat - Car Italian?

Yeah.

So, in some cases s there are also.

SpeechDat - Car Italian. Yeah. There're a few cases, actually, which I found later, that there are.

o Uh - huh.

So that was one of the bugs that was there in estimating the noise. And, uh, so once it was cleared, uh, I ran a few experiments with different ways of smoothing the estimated clean speech and how t estimated the noise and, eh, smoothing the SNR also. And so the the trend seems to be like, uh, smoothing the current estimate of the clean speech for deriving the SNR, which is like deriving the Wiener filter, seems to be helping. Then updating it quite fast using a very small time constant. So we'll have, like, a few results where the estimating the the More smoothing is helping. But still it's like it's still comparable to the baseline. I haven't got anything beyond the baseline. But that's, like, not using any Wiener filter. And, uh, so I'm I'm trying a few more experiments with different time constants for smoothing the noise spectrum, and smoothing the clean speech, and smoothing SNR. So there are three time constants that I have. So, I'm just playing around. So, one is fixed in the line, like Smoothing the clean speech is is helping, so I'm not going to change it that much. But, the way I'm estimating the noise and the way I'm estimating the SNR, I'm just trying trying a little bit. So, that h And the other thing is, like, putting a floor on the, uh, SNR, because that if some In some cases the clean speech is, like when it's estimated, it goes to very low values, so the SNR is, like, very low. And so that actually creates a lot of variance in the low - energy region of the speech. So, I'm thinking of, like, putting a floor also for the SNR so that it doesn't vary a lot in the low - energy regions. And, uh. So. The results are, like So far I've been testing only with the baseline, which is which doesn't have any LDA filtering and on - line normalization. I just want to separate the the contributions out. So it's just VAD, plus the Wiener filter, plus the baseline system, which is, uh, just the spectral I mean, the mel sp mel, uh, frequency coefficients. Um. And the other thing that I tried was but I just took of those, uh, Carlos filters, which Hynek had, to see whether it really h helps or not. I mean, it was just a a run to see whether it really degrades or it helps. And it's it seems to be like it's not hurting a lot by just blindly picking up one filter which is nothing but a four hertz a band - pass m m filter on the cubic root of the power spectrum. So, that was the filter that Hy - uh, Carlos had. And so Yeah. Just just to see whether it really it's it's is it worth trying or not. So, it doesn't seems to be degrading a lot on that. So there must be something that I can that can be done with that type of noise compensation also, which I guess I would ask Carlos about that. I mean, how how he derived those filters and and where d if he has any filters which are derived on OGI stories, added with some type of noise which what we are using currently, or something like that. So maybe I'll.

This is cubic root of power spectra?

Yeah. Cubic root of power spectrum.

So, if you have this band - pass filter, you probably get n you get negative values. Right?

Yeah. And I'm, like, floating it to z zeros right now.

OK.

So it has, like the spectrogram has, like Uh, it actually, uh, enhances the onset and offset of I mean, the the begin and the end of the speech. So it's there seems to be, like, deep valleys in the begin and the end of, like, high - energy regions,

Mm - hmm.

because the filter has, like, a sort of Mexican - hat type structure.

Mm - hmm.

So, those are the regions where there are, like when I look at the spectrogram, there are those deep valleys on the begin and the end of the speech. But the rest of it seems to be, like, pretty nice.

Mm - hmm.

So. That's something I observe using that filter. And Yeah. There are a few very not a lot of because the filter doesn't have a really a deep negative portion, so that it's not really creating a lot of negative values in the cubic root. So, I'll I'll s may continue with that for some w I'll I'll Maybe I'll ask Carlos a little more about how to play with those filters, and but while making this Wiener filter better. So. Yeah. That that's it, Morgan.

Uh, last week you were also talking about building up the subspace stuff?

Yeah. I I I would actually m m didn't get enough time to work on the subspace last week. It was mostly about finding those bugs and

OK.

th you know, things, and I didn't work much on that.

How about you, Carmen?

Well, I am still working with, eh, VTS. And, one of the things that last week, eh, say here is that maybe the problem was with the diff because the signal have different level of energy.

Hmm?

And, maybe, talking with Stephane and with Sunil, we decide that maybe it was interesting to to apply on - line normalization before applying VTS. But then we decided that that's it doesn't work absolutely, because we modified also the noise. And Well, thinking about that, we we then we decide that maybe is a good idea. We don't know. I don't hav I don't this is I didn't do the experiment yet to apply VTS in cepstral domain.

The other thing is So so, in i i and Not and C - zero would be a different So you could do a different normalization for C - zero than for other things anyway. I mean, the other thing I was gonna suggest is that you could have two kinds of normalization with with, uh, different time constants. So, uh, you could do some normalization s uh, before the VTS, and then do some other normalization after. I don't know. But but C - zero certainly acts differently than the others do,

Uh.

so that's.

Mm - hmm.

Well, we s decide to m to to obtain the new expression if we work in the cepstral domain. And Well. I am working in that now,

Uh - huh.

but I'm not sure if that will be usefu useful. I don't know. It's k it's k It's quite a lot It's a lot of work.

Uh - huh.

Well, it's not too much, but this it's work.

Yeah.

And I want to know if if we have some feeling that the result I I would like to know if I don't have any feeling if this will work better than apply VTS aft in cepstral domain will work better than apply in m mel in filter bank domain. I r I'm not sure. I don't I don't know absolutely nothing.

Mm - hmm.

Yeah. Well, you're I think you're the first one here to work with VTS, so, uh, maybe we could call someone else up who has, ask them their opinion. Uh,

Mm - hmm.

I don't I don't have a good feeling for it. Um.

Pratibha.

Actually, the VTS that you tested before was in the log domain and so the codebook is e e kind of dependent on the level of the speech signal.

Yeah?

And So I expect it If if you have something that's independent of this, I expect it to it to, uh, be a better model of speech.

To have better.

And. Well.

You you wouldn't even need to switch to cepstra. Right? I mean, you can just sort of normalize the.

No. We could normali norm I mean, remove the median.

Yeah. Yeah. And then you have one number which is very dependent on the level cuz it is the level,

Mm - hmm.

and the other which isn't.

Mm - hmm. Yeah. But here also we would have to be careful about removing the mean of speech and not of noise.

Ye

Because it's like first doing general normalization

Yea

and then noise removal, which is.

Yeah. We I was thinking to to to estimate the noise with the first frames and then apply the VAD,

Mm - hmm.

Mm - hmm.

before the on - line normalization.

Mm - hmm.

We we see Well, I am thinking about that and working about that,

Yeah.

but I don't have result this week.

Sure. I mean, one of the things we've talked about maybe it might be star time to start thinking about pretty soon, is as we look at the pros and cons of these different methods, how do they fit in with one another? Because we've talked about potentially doing some combination of a couple of them. Maybe maybe pretty soon we'll have some sense of what their characteristics are,

Mm - hmm.

so we can see what should be combined.

Mm - hmm.

Is that it? OK?

OK. Why don't we read some digits?

Yep. Want to go ahead, Morgan?

Sure.

Transcript L dash two one five.

O K.

Question: What major tasks or decisions needed to be completed after the meeting?

Response: The meeting transcript does not contain any information about major tasks or decisions needed to be completed after the meeting.

Question: Did anyone take issue with Barry's assertions about intermediate categories?

Response: The meeting transcript does not contain any information about Barry's assertions on intermediate categories, nor any issues being raised regarding this topic.

Question: What did Prof. Hermansky recommend to do when assessing the effectiveness of mean subtraction as a technique?

Response: There is no participant named Prof. Hermansky in the meeting.

Question: What thoughts did Dr. Avendano share with the participants regarding hybrid systems?

Response: The meeting transcript does not have any information about Dr. Avendano sharing thoughts on hybrid systems.

Question: Can you give a summary on the algorithmic discussion about the spectrograms?

Response: I am sorry, but the provided portion of the meeting transcript does not include any discussions concerning spectrograms. Therefore, I am unable to provide a summary of the algorithmic discussion about the spectrograms.

Question: Did any participants voice disagreements with Prof. B's perspective on the Wiener filter?

Response: I am sorry, but the provided portion of the meeting transcript does not contain any discussions regarding the Wiener filter. Therefore, I am unable to provide the answer.

Question: What made working with the G-712 filter advantageous?

Response: I am sorry, but the provided portion of the meeting transcript does not contain any information regarding the advantages of working with the G-712 filter. Therefore, I am unable to provide the answer.

Question: What made working with the G-712 filter beneficial?

Response: The G-712 filter was not mentioned in the transcript, thus, its advantages or benefits cannot be determined from the conversation.

Question: Did Prof. Hermansky recommend using the same parameters for training and testing the SRI system?

Response: There is no participant named Prof. Hermansky in the meeting.

Question: What were his reasons for this recommendation?

Response: There is no participant named Prof. Hermansky in the meeting.


How many batteries do you go through?

Thank you.

Alright.

Sure.

Good. Yeah. OK so, let's get started. Nancy said she's coming and that means she will be. Um. My suggestion is that Robert and Johno sort of give us a report on last week's adventures uh to start. So everybody knows there were these guys f uh from Heidelber - uh, uh, actually from uh DFKI uh, part of the German SmartKom project, who were here for the week and, I think got a lot done.

Yeah, I think so too. Um. The we got to the point where we can now speak into the SmartKom system, and it'll go all the way through and then say something like " Roman numeral one, am Smarticus. " It actually says, " Roemisch einz, am Smarticus, "

OK.

which means it's just using a German sythesis module for English sentences.

OK.

So uh,

It doesn't know " I ".

OK.

Um, the uh

Oh, Am Spartacus. "

" I am Sm - I am Smarticus " is what it's saying.

Right.

Verstehe. OK.

I gue

The uh sythesis is just a question of um, hopefully it's just a question of exchanging a couple of files, once we have them. And, um, it's not going to be a problem because we decided to stick to the so - called concept to speech approach. So I'm I'm I'm going backwards now, so " synthesis " is where you sort of make this uh, make these sounds, and " concept to speech " is feeding into this synthesis module giving it what needs to be said, and the whole syntactic structure so it can pronounce things better, presumably. Then, just with text to speech.

Mm - hmm.

And, uh, Johno learned how to write XML tags. Uh, and did write the tree adjoining grammar for some some sentences. No, right?

Yeah.

Yeah, for a couple.

So. Bu - Uh, i The way the uh, the dialogue manager works is it dumps out what it wants to know, or what it wants to tell the person, to a er in XML and there's a conversion system for different uh, to go from XML to something else. And th so, the knowledge base for the system, that generates the syntasti syntactic structures for the ge generation is uh, in a LISP - like the knowledge base is in a LISP - like form. And then the thing that actually builds these syntactic structures is something based on Prolog. So, you have a basically, a goal and it, you know, says " OK, well I'm gonna try to do the Greet - the - person goal,

Mm - hmm.

so it just starts uh, it binds some variables and it just decides to, you know, do some subscold. Basically, it just means " build the tree. "

OK.

And then it passes the tree onto, uh, the ge the generation module.

But I think that the point is that out of the twelve possible utterances that the German system can do, we've already written the the syntax trees for three or four.

We yeah. So, the syntax trees are very simple. It's like most of the sentences in one tree,

Mm - hmm.

and instead of, you know, breaking down to, like, small units and building back up, they basically took the sentences, and basically cut them in half, or you know, into thirds or something like that, and made trees out of those. And so uh, uh Tilman wrote a little tool that you could take LISP notation and generate an XML, uh, tree. Uh, S what do ca structure from the from the LISP. And so basically you just say, you know, " noun goes to ", you know, Er, nah, I don't re I've never been good at those. So there's like the VP goes to N and those things in LISP, and it will generate for you.

OK. N, N, V yeah, OK. Alright.

And because we're sticking to that structure, the synthesis module doesn't need to be changed. So all that f fancy stuff, and the Texas speech version of it, which is actually the simpler version, is gonna be done in October which is much too late for us. So. This way we we worked around that. The, uh the system, um I can show you the system. I actually want, at least, maybe, you should be able to start it on your own. If you wanna play around with it, in th in the future. Right now it's brittle and you need to ch start it up and then make ts twenty changes on on on on seventeen modules before they actually can stomach it, anything. And send in a a a couple of side queries on some dummy center set - up program so that it actually works because it's designed for this seevit thing, where you have the gestural recognition running with this s Siemens virtual touch screen, which we don't have here.

Mm - hmm.

And so we're doing it via mouse, but the whole system was designed to work with this thing and it was It was a lot of engineering stuff. No science in there whatsoever, but it's working now, and um, that's the good news. So everything else actually did prove to be language independent except for the parsing and the generation.

Why I had I did need to chan generate different trees than the German ones, mainly because you know like uh, the gerund in in German is automatically taken care of with just a regular verb,

You have to switch it on.

Mm - hmm.

so I'd uh have to add " am walking, "

OK.

or I'd have to add a little stem for the " am ", when I build the built the tree.

OK. Yeah, I noticed that um, that some of the examples they had, had you know, non - English word orders and so on, you know. And then all that good stuff. So.

Alright.

Yeah.

Like.

So it might be worth, Keith, you looking at this,

Yeah.

um

I I still don't I still don't really understand e like.

Well Tilman s

I mean we sort of say, um You know, I I still don't exactly understand sort of the information flow uh in in this thing, or what the modules are and so on. So, you know, like just that such - and - such module uh um decides that it wants to achieve the goal of greeting the user, and then magically it sort of s

Yeah.

I mean, how does it know which syntactic structure to pull out, and all that?

I thi Yeah. So. I think it's not worth going over in the group,

R uh Sure.

but sort of when you get free and you have the time uh either Robert or Johno or I can walk you through it.

Yeah, soon. OK.

And you can ask all the questions about how this all fits together.

That's fine.

It's eee  messy but once you understand it you understand it. It's it's There's nothing really complicated about it.

OK.

No.

And I remember one thing that that came up in the talk last Wednesday. Um, was this, I I think he talked about the idea of like, um He was talking about these lexicalized uh, uh, tree adjoining grammars where you sort of for each word you, um.

OK, you know how to do it?

For each lexical item, the lexical entry says what all the uh trees are that it can appear in. And of course, that's not v That's the opposite of constructional. That's, you know, that's that's HPSG or whatever.

Right.

You know?

Right. Now, we're we're not committed for our research to do any of those things.

Yeah. Mm - hmm.

So uh we are committed for our funding.

Right.

OK? to uh.

Make our stuff fit to that.

Yeah, to n no, to just get the dem get the demos they need.

Uh - huh.

OK? So between us all we have t to get th the demos they need. If it turns out we can also give them lots more than that by, you know, tapping into other things we do, that's great.

You should probably move the microphone closer to your face.

Mm - hmm.

But i it turns out not to be in an any of the contracts

There's like a little The twisty thing, you can move it with.

OK.

and, s deliberately. So, the reason I'd like you to understand uh what's going on in this demo system is not because it's important to the research. It's just for closure. So that if we come up with a question of " could we fit this deeper stuff in there? " or something. You know what the hell we we're talking about fitting in.

Right. OK.

So it's just, uh in the sam same actually with the rest of us we just need to really understand what's there. Is there anything we can make use of? Uh, is there anything we can give back, beyond th the sort of minimum requirements? But none of that has a short time fuse.

OK.

So th the demo the demo requirements for this Fall are sort of taken care of as of later this week or something. And then So, it's probably fifteen months or something until there's another serious demo requirement.

Oh OK.

That doesn't mean we don't think about it for fifteen months,

Right.

but it means we can not think about it for six months.

Right, yeah.

So. The plan for this summer uh, really is to step back from the applied project,

Right.

keep the d keep the context open, but actually go after the basic issues.

Hmm. Oh OK.

And, so The idea is there's this uh, other subgroup that's worrying about formalizing the nota getting a notation. But sort of in parallel with that, uh, the hope is tha in particularly you will work on constructions in English Ge - and German for this domain,

Mm - hmm.

but y not worry about parsing them or fitting them into SmartKom or any of the other anything lik any other constraints for the time being.

Yeah. OK. Got it.

It's hard enough to get it semantically and syntactically right and then and get the constructions in their form and stuff.

Yeah.

And, I don I don't want you f feeling that you have to somehow meet all these other constraints.

Right, OK.

Um. And similarly with the parsing, uh we're gonna worry about parsing uh, the general case you know, construction parser for general constructions. And, if we need a cut - down version for something, or whatever, we'll worry about that later.

OK.

So I'd like to, for the summer turn into science mode.

OK.

And I assume that's also, uh, your plan as well.

So I mean, the the point is that like the meetings um so far that I've been at have been sort of been geared towards this demo,

Right. Yeah. Yeah.

and then that's going to go away pretty soon.

But but we we're swit

OK.

Right.

And then we'll sort of shift gears a Fairly substantially,

Yeah.

It's.

Yeah.

huh?

Yeah.

It's got. What I what I think is is a good idea that I can can show to anyone who's interested, we can even make a sort of an internal demo, and I I show you what I do,

Mm - hmm.

I speak into it and you hear it talk,

OK.

and I can sort of walk f through the information. So, this is like in half hour or forty - five minutes. Just fun.

OK.

And so you when somebody on the streets com comes up to you and asks you what is SmartKom so you can, sort of, give a sensible answer.

Right. OK.

So, c sh we could set that up as actually an institute wide thing? Just give a talk in the big room, and and so peo people know what's going on? when you're ready?

Absolutely.

Yeah I mean, that's the kind of thing That's the level at which you know we can just li invite everybody and say " this is a project that we've been working on and here's a demo version of it " and stuff like that.

Yeah.

OK. Well d we we do wanna have all the bugs out b where you have to sort of pipe in extra XML messages from left and right before you're.

Uh - huh.

Indeed.

Yeah. OK. Makes sense.

But any so that e e It's clear, then, I think. Actually, roughly starting uh let's say, nex next meeting, cuz this meeting we have one other thing to tie up besides the trip report.

Yeah. OK.

But uh starting next meeting I think we want to flip into this mode where Uh. I mean there are a lot of issues, what's the ontology look like,

Mm - hmm.

you know what do the constructions look like, what's the execution engine look like, mmm lots of things.

Mm - hmm.

But, more focused on uh an idealized version than just getting the demo out. Now before we do that, let's get back in Oh! But, it's still, I think, useful for you to understand the demo version enough, so that you can can see what what it is that that uh it might eventually get retro - fitted into or something.

Yeah. OK, right.

And Johno's already done that, uh, looked at the dem uh the looked at the SmartKom stuff.

Wa uh To some de uh what what part of th the SmartKom stuff?

Well, the parser, and that stuff.

Oh yeah yeah.

OK. Anyway. So, the trip the report on these the last we we sort of interrupted you guys telling us about what happened last week.

Yeah. It's alright.

Um. Well it was just amazing to to see uh how how instable the whole thing is,

Maybe you're done, then.

and if you just take the And I g I got the feeling that we are the only ones right now who have a running system. I don't know what the guys in Kaiserslautern have running because e the version that is, the full version that's on the server d does not work. And you need to do a lot of stuff to make it work. And so it's And even Tilman and Ralf sort of said " yeah there never was a really working version that uh did it without th all the shortcuts that they built in for the uh October @ @ version ". So we're actually maybe ahead of the System Gruppe by now, the system the integration group. And it was, uh It was fun to some extent, but the uh the outcome that is sort of of scientific interest is that I think both Ralf and Tilman um, I know that they enjoyed it here, and they r they they liked, uh, a lot of the stuff they saw here, what what we have been thinking about, and they're more than willing to to um, cooperate, by all means. And um, part of my responsibility is uh to use our internal " group - ware " server at EML, make that open to all of us and them, so that whatever we discuss in terms of parsing and and generating and constructions w we we sort of uh put it in there and they put what they do in there and maybe we can even um, get some overlap, get some synergy out of that. And um, the, uh If I find someone at in EML that is interested in that, um I I may even think that we could look take constructions and and generate from them because the tree adjoining grammars that that Tilman is using is as you said nothing but a mathematical formalism. And you can just do anything with it, whether it's syntactic trees, H P S G - like stuff, or whether it's construction. So if you ever get to the generation side of constructing things and there might be something of interest there, but in the moment we're of course definitely focused on the understanding, um, pipeline.

Anyth - any other  uh repo visit reports sort of stories? uh we so we now know I think, what the landscape is like.

Mm - hmm.

And so we just push on and and uh, do what we need to do. And one of the things we need to do is the um, and this I think is relatively tight tightly constrained, is to finish up this belief - net stuff. So. Uh. And I was going to switch to start talking about that unless there're m other more general questions. OK so here's where we are on the belief - net stuff as far as I understand it. Um. Going back I guess two weeks ago uh Robert had laid out this belief - net, missing only the connections. Right? That is  So, he'd put all th all the dots down, and we went through this, and, I think, more or less convinced ourselves that at least the vast majority of the nodes that we needed for the demo level we were thinking of, were in there. Yeah  we may run across one or two more. But of course the connections weren't. So, uh Bhaskara and I went off and looked at some technical questions about were certain operations sort of legitimate belief - net computations and was there some known problem with them or had someone already uh, solved you know how to do this and stuff. And so Bhaskara tracked that down. The answer seems to be uh, " no, no one has done it, but yes it's a perfectly reasonable thing to do if that's what you set out to do ". And, so the current state of things is that, again, starting now, um we'd like to actually get a running belief - net for this particular subdomain done in the next few weeks. So Bhaskara is switching projects as of the first of June, and uh, he's gonna leave us an inheritance, which is a uh hopefully a belief - net that does these things. And there're two aspects to it, one of which is, you know, technical, getting the coding right, and making it run, and uh stuff like that. And the other is the actual semantics. OK? What all you know, what are the considerations and how and what are the ways in which they relate. So he doe h he doesn't need help from this group on the technical aspects or if he does uh we'll do that separately.

Mm - hmm.

But in terms of what are the decisions and stuff like that, that's something that we all have to work out. Is is that right? I mean that's that's both you guys' understanding of where we are?

Absolutely.

OK.

So, I guess, um Is there like a latest version of the belief - net of the proposed belief - net? Like.

We had um decided.

like.

Um. Well, no, we didn't decide. We wanted to look into maybe getting it, the visualization, a bit clearer, but I think if we do it, um, sort of a paper version of all the nodes and then the connections between them, that should suffice.

Mm - hmm. Yeah, that should be fine.

Yeah I mean, that's a separate problem.

Yeah, I.

We do in the long run wanna do better visualization and all that stuff.

Yeah.

That's separable, yeah.

I did look into that, uh in terms of, you know, exploding the nodes out and down ag

Yep. Right.

JavaBayes does not support that. I can imagine a way of hacking at the code to do that. It'd probably take two weeks or so to actually go through and do it,

Not not at this point.

and I went through all the other packages on Murph - Kevin Murphy's page,

Right.

and I couldn't find the necessary mix of free and uh with the GUI and, with this thing that we want.

Well, we can p If it's If we can pay Yeah. If you know it's paying a thousand dollars or something we can do that. OK? So so don't view free as as a absolute constraint.

OK. OK, so then I'll go back and look at the ones on the list that.

OK. And you can ask Kevin.

But.

Yeah.

Mmm.

But.

Yeah, the one that uh people seem to use is uh Hugin or whatever?

Hugin, yeah that's free.

How exp I don't think it's Is it free? Because I've seen it advertised in places so I it seems to.

Uh, it may be free to academics. Like I I don't know. I have a co  I have a copy  that I l I downloaded.

OK.

So, at one point it was free.

OK.

Uh but yo I noticed people do use Hugin so um,

How do you spell that?

HUGIN.

Why

And Bhaskara can give you a pointer. So then, in any case, um But paying a lit You know, if i if it's uh Probably for university, it's it's gonna be real cheap anyway. But um, you know, if it's fifty thousand dollars we aren't gonna do it. I'm mean, we have no need for that.

I I also s would suggest not to d spend two weeks in in in changing the the JavaBayes code.

No,

Yeah.

he's not gonna do that.

OK.

I I will send you a pointer to a Java applet that does that, it's sort of a fish - eye. You you have a node, and you click on it, and it shows you all the connections,

Mmm.

and then if you click on something else that moves away, that goes into the middle. And maybe there is an easy way of interfacing those two. If that doesn't work, it's not a problem we we need to solve right now. What I'm what my job is, I will, um, give you the input in terms of of the internal structure. Maybe node by node, or something like this? Or should I collect it all

Mm - hmm.

and.

Doesn't matter.

Um, just any like like sort of rough representation of the entire belief - net is probably best.

OK. And um you're gonna be around? t again, always Tuesdays and Thursdays afternoon - ish? As usual? Or will that change?

Yeah I mean, yeah, I can like I c Um. This week I guess um, kind of I have a lot of projects and stuff but after that I will generally be more free. So yes, I might I can be around. And g I mean, generally if you email me also I can be around on other days.

Yeah. OK.

Yeah and this is not a crisis that I mean, you do, e everybody who's a student should, you know do their work, get their c courses all in good shape and and and and then we'll dig d dig down on this.

Yeah, that's Yeah. OK. No, that's good. That means I have I h I can spend this week doing it. So.

OK.

How do you go about this process of deciding what these connections are? I know that there's an issue of how to weight the different things too, and stuff. Right? I mean do you just sort of guess and see if it sort of.

Right. Well there there there There're two different things you do.

It's.

One is you design and the other is you learn. OK? So uh what we're gonna do initially is is do design, and, i if you will, guess.

OK.

OK. Uh that is you know use your best knowledge of of the domain to uh, hypothesize what the dependencies are and stuff.

Right. OK.

If it's done right, and if you have data then, there are techniques for learning the numbers given the structure

Yeah.

and there are even techniques for learning the structure, although that takes a lot more data, and it's not as @ @ and so forth and so on. So uh but for the limited amount of stuff we have for this particular exercise I think we'll just design it.

Alright.

Yeah. Fo - Hopefully as time passes we'll get more and more data from Heidelberg and from people actually using it and stuff.

OK.

So but this is the  long run.

Yeah.

But to solve our problems ag uh a mediocre design will do I think in the beginning.

Yeah, that's right. Yeah, oh, and by the way, speaking of data, um, are there I could swore uh, I could swear I saw it sitting on someone's desk at some point, but is there a um a transcript of any of the, sort of, initial interactions of people with the with the system? Cuz you know, I'm still sort of itching to to look at what look at the stuff, and see what people are saying.

Mm - hmm. Yeah. Yeah make yourself a note. So and and, of course Keith would like the German as well as the English, so whatever you guys can get.

The German. Oh yeah, of course, German. Yeah.

Yeah, the y your native language, right? You remember that one.

OK. That's important, yeah.

Yeah,

So he'll get you some data.

u OK. Yeah, I mean I I sort of um found the uh, uh the audio of some of those, and um, it kind of sounded like I didn't want to trudge through that, you know.

Hmm.

It was just Strange, but.

Yep.

We probably will not get those to describe because they were trial runs.

Oh yeah, OK.

Um, but uh that's th but we have data in English and German already.

OK, yeah, I mean.

So. Transcribed. I will send you that. OK.

OK, so while we're still at this sort of top level, anything else that we oughta talk about today?

Ho - how was your thingy.

Oh, um, I just wanted to, uh, s like mention as an issue, um, you know last meeting I wasn't here because I went to a linguistics colloquium on the fictive motion stuff,

Oh right.

and that was pretty interesting and you know, I mean, seems to me that that will fairly obviously be of relevance to uh to what we're doing here because you know people are likely to give descriptions like you know, " What's that thing uh right where you start to go up the hill, " or something like that, you know, meaning a few feet up the hill or whatever from some reference point and all that stuff so I mean, I'm sure in terms of you know, people trying to state locations or, you know, all that kind of stuff, this is gonna be very relevant. So, um, now that was the talk was about English versus Japanese, um, which obviously the Japanese doesn't affect us directly, except that, um, some of the construction he'd what he talked about was that you know in English we say things like th you know, " your bike is parked across the street " and we use these prepositional phrases, you know, " well, if you were to move across the street you would be at the bike ", but um in in Japanese the the more conventionalized tendency is to use a sort of a description of " where one has crossed to the river, there is a tree ". Um, and you know, you can actually say things like, um, " there's a tree where one has crossed the river, but no one has ever crossed the river ", or something like that. So the idea is that this really is you know that's supposed show that's it's really fictive and so on. But um But the point is that that kind of construction is also used in English, you know, like " right where you start to go up the hill ", or " just when you get off the train ", or something like that to uh, to indicate where something is.

Mmm.

So we'll have to think about.

So how much is that used in German?

Um. The uh Well I wa I was on a uh on a on a different sidetrack.

Oh, OK.

I mean, the the Deep Map project which um is undergoing some renovation at at the moment, but this is a a three language project : German, English, Japanese.

OK.

And um, we have a uh, uh I have taken care that we have the the Japanese generation and stuff. And so I looked into uh spatial description. So we can generate spatial descriptions, how to get from A to B. And and information on objects, in German, English, and Japanese.

Mm - hmm.

And there is a huge uh project on spatial descriptions uh differences in spatial descriptions. Well, if yo if you're interested in that, so how how, I mean it does sort of go d all the way down to the conceptual level to some extent.

OK.

So. Um.

So, where is this huge project?

It's KLEIST. It's the uh Bielefeld generation of uh spatial descriptions and whatever.

Mm - hmm.

Well, that may be another thing that Keith wants to look at.

OK.

But um, I I think we should leave Japanese constructions maybe outside of the scope for for now,

Yeah.

but um definitely it's interesting to look at at cross the bordered there.

Mm - hmm.

Are are you going to p pay any attention to the relative position of of the direction relative relative to the speaker? For example, there are some differences between Hebrew and English. We can say um " park in front of the car " as you come beh you drive behind the car. In Hebrew it means " park behind the car ", because to follow the car is defined as it faces you.

Mm - hmm. Intrinsic, yeah.

While in English, front of the car is the absolute front of the car.

OK.

So.

Right, so the canonical direction of motion determines where the front is.

Right. Right.

OK.

So, i i i is German uh closer to to E uh, uh, uh, uh to E I mean uh

Mm - hmm.

I don't think it it's related to syntax, though, so it may be entirely different.

Um, as a matter of fact.

No, it's not.

Right.

Yeah.

Um. Did you ever get to look at the the rou paper that I sent you on the on that problem in English and German?

I think.

Carroll, ninety - three. Um. I There is a a study on the differences between English and German on exactly that problem.

Hmm.

So it's they actually say " the monkey in front of the car, where's the monkey? "

Mm - hmm.

And, um, they found statistically very significant differences in English and German, so I I I It might be, since there are only a finite number of ways of doing it, that that German might be more like Hebrew in that respect.

Hmm.

The solution they proposed was that it was due to syntactic factors.

That but it wasn't was.

That syntactic facto factors do do play a role there, wh whether you're more likely, you know, to develop uh, choices that lead you towards using uh intrinsic versus extrinsic reference frames.

Right. Mm - hmm. Right.

I mean, it seems to me that you can get both in in English depending o

Hmm.

You know, like, " in front of the car " could you know Like, here's the car sideways to me in between me and the car or something's in front of the car, or whatever. I could see that,

Absolutely.

but But anyway, so you know, I mean, this was this was a a very good talk on those kinds of issues and so on. So uh.

I can also give you uh, a pointer to a paper of mine which is the the ultimate taxonomy of reference frames.

Alright! Cool!

So.

Oh.

I'm the only person in the world who actually knows how it works.

Oh.

Not really.

Great. No, I've not seen that.

What do you mean. Um. " reference frames "?

It's called a.

uh uh

It's it's spatial reference frames. You actually have only Um. If you wanna have a This is usually um I should there should be an " L ", though. Well actually you have only have two choices. You can either do a two - point or a three - point which is you You're familiar with th with the " origo "? where that's the center " Origo " is the center of the f frame of reference.

Hmm.

And then you have the reference object and the object to be localized.

Hmm.

Mm - hmm.

OK? In some cases the origo is the same as the reference object.

So that would be " origin " in English,

This was like.

The origin.

Right

Yeah.

right?

" Origo " is a Terminus technikus. in that sense, that's even used in the English literature. " Origo. "

Oh, OK. I never heard it.

Alright.

OK.

OK.

And um, so, this video tape is in front of me.

Mm - hmm.

I'm the origo and I'm also the reference object.

Mm - hmm.

Right.

Those are two - point.

Mm - hmm.

And three - point relations is if something has an intrinsic front side like this chair then your f shoe is behind the chair.

Yeah.

Mm - hmm.

And, reference object and Um. No, from from my point of view your shoe is left of the chair.

Right. You you can actually say things like, um, " it's behind the tree from me " or something like that, I think, in in in certain circumstances in English, right? As sort of " from where I'm standing it would appear that ".

Yeah. Yeah. So,

Looks a little bit like Reichenbach for time.

Yeah, it sounds like it, doesn't it,

Yeah.

yeah.

It's a lot like it.

And then and then here you.

Um.

On this scale, you have it either be ego or allocentric.

Mm - hmm.

And that's  that's basically it. So. Egocentric two - point, egocentric three - point, or you can have allocentric.

Oh, OK.

So, " as seen from the church, the town hall is right of that um, fire station ". aa - huh  It's hardly ever used but it's w

I'd love to see it if you if you have a copy kind of. Uh.

Yeah.

Yeah. I see this is this is getting into Ami's thing.

Here

Mm - hmm.

He's he's very interested in that.

OK.

So.

Me too.

Uh. Yeah. Well, why don't you just put it on the web page? There's this EDU Right?

Yeah it's or or just Yeah.

Or a link to it.

It's also all on my my home page at EML. It's called " An Anatomy of a Spatial Description ".

Just

But I'll send that link.

OK, great.

Maybe just put a link on. Yeah.

Yep.

By the way, there something that I didn't know until about a week ago or so, is apparently, there are separate brain areas for things within reach, and things that are out of reach.

Yep.

Huh.

So there's there's uh all this linguistic stuff about you know, near and far, or yon and and so forth.

Mm - hmm.

So this is all This is There's this linguistic facts. But apparently, the Uh. Here's the way the findings go. That, you know they do MRI, and and if you're uh got something within reach then there's one of your areas lights up, and if something's out of reach uh a different one. But here's the the amazing result, um, they say. You get someone with a with a deficit so that they have a perfectly normal ability at distance things. So the s typical task is subdivision. So there's a a line on the wall over there, and you give them a laser pointer, and you say, " Where's the midpoint? " And they do fine. If you give them the line, and they have to touch it, they can't. There's just that part of the brain isn't functioning, so they can't do that. Here's the real experiment. The same thing on the wall, you give them a laser, " where is it? ",

Mm - hmm.

they do it. Give them a stick, long stick, and say " do it ", they can't do it. So there's a remapping of distant space into nearby space.

Right. So they doubled the the end the end of this.

Because it's within reach now?

Yeah,

It's not within reach and you use the Within - Reach uh, mechanism.

yeah.

Oh. Wow.

Circuits.

Right.

So I'll d I'll dig you up this reference.

That's cool.

And so this doe This is, uh First of all, it explains something that I've always wondered about and I'll do this this test on you guys as well. So. Uh. How - I have had an experience, not often, but a certain number of times, when, for example, I'm working with a tool, a screwdriver or something, for a long time, I start feeling the tip directly. Not indirectly, but you actually can feel the tip.

Yeah yeah.

And people who are uh accomplished violinists and stuff like that, claim they also have this kind of thing where you get a direct sensation of, physical sensation, of the end affector.

Yeah. What's going on at the end of the tool,

The ext the the The extension,

yeah.

Huh?

What's going on at the end of the tool, or whatever.

Yeah, within.

right.

Huh?

The extension of of your hand, right.

Yeah, right. Have you hav y h had this?

The I I think so. I mean i i it's not exactly the th same thing, but but s it it it's getting close to that.

Yeah.

W what does it feel like?

Oh i it feels like your as if your uh neurons had extended themselves out to this tool, and you're feeling forces on it and so forth and and you deal directly with it.

I once I I was playing you know with those um uh devices that allow you to manipulate objects when it's dangerous to get close? So you can insert your hand something

Oh, OK.

Right, yeah yeah yeah. Yeah.

and there's a correspondence between.

Yeah.

So I played with it. After a while, you don't feel the difference anymore. I I mean it's kind of.

Mm - hmm.

Yeah, right.

Very kind of you stop back and suddenly it goes away and you have to kind of work again to recapture it, but yeah.

Yeah.

Right, Yeah, so anyway, so So this was the first actual experimental evidence I'd seen that was consistent with this anecdotal stuff.

That's cool.

And of course it makes a lovely def uh story about why languages uh, make this distinction. Of course there are behavioral differences too. Things you can reach are really quite different than things you can't.

Yeah.

But there seems to be an actu really deep embodied neural difference. And i this is, um So. In addition to the e

This is more proximal - distal.

Yeah uh exactly. So in addition to e ego and allocentric uh which appear all over the place, you also apparently have this proximal - distal thing which is very deeply uh embedded. S

Well, Dan Montello sort of, he he does the uh uh th the cognitive map world, down in Santa Barbara. And he he always talks about these He he already well i probably most likely without knowing this this evidence uh is talking about these small scale spaces that you can manipulate versus large scale environmental spaces.

Yeah. Well there's there's uh been a lot of behavioral things o on this, but that was the first neur neuro - physiological thing I saw. Anyway yeah, so we'll we'll look at this. And. So, all of these issues now are now starting to come up. So, now we're now done with demos. We're starting to do science, right? And so these issues about uh, reference, and spatial  reference, discourse reference, uh - uh - uh - uh  all this sort of stuff, uh, deixis which is part of what you were talking about,

Mm - hmm. Mm - hmm.

um So, all of this stuff is coming up essentially starting now. So we gotta do all this. So there's that. And then there's also a set of system things that come up. So " OK, we're not using their system. That means we need our system. "

Mm - hmm.

Right?

Yeah.

It it follows. And so, uh, in addition to the business about just getting the linguistics right, and the formalism and stuff, we're actually gonna build something and uh, Johno is point person on the parser, analyzer, whatever that is, and we're gonna start on that in parallel with the um, the grammar stuff.

Alright.

But to do that we're gonna need to make some decisions like ontology, so, um And so this is another thing where we're gonna, you know, have to get involved and make s relatively early I think, make some decisions on uh, " is there an ontology API that that " There's a sort of standard way of getting things from ontologies and we build the parser and stuff around that, or is there a particular ontology that we're gonna standardize on, and if so For example, is there something that we can use there. i Does uh either the uh SmartKom project or one of the projects at EML have something that we can just p pull out, for that. Uh, so there are gonna be some some some things like that, which are not science but system. But we aren't gonna ignore those cuz we're we're not only going The plan is not only to lay out this thing, but to actually uh build some of it. And how much we build, and and so forth.

I.

Uh. Part of it, if it works right, is wh It looks like we're now in a position that the construction analyzer that we want for this applied project can be the same as the construction analyzer that Nancy needs for the child language modeling. So. It's always been out of phase but it now seems that um, there's a good shot at that. So we've talked about it, and the hope is that we can make these things the same thing,

OK.

and of course it's only w In both cases it's only one piece of a bigger system.

Mm - hmm.

But it would be nice if that piece were exactly the same piece.

Right.

It was just this uh construction analyzer. And so we think we think we have a shot at at that.

OK.

So. The for So. To to come full circle on that, this formalization task, OK? is trying to get the formalism into into a shape where it can actually uh

Yeah. Be of use to someone who's trying to do this, right?

d Well, yeah, where it actually is is covers the whole range of things. And the the the the thing that got Mark into the worst trouble is he had a very ambitious thing he was trying to do, and he insisted on trying to do it with a limited set of mechanisms. It turned out, inherently not to cover the space.

OK.

And it just it was just terribly frustrating for him,

Yeah.

and he seemed fully committed to both sides of this i i irreconcilable thing.

I see. Right.

And. Uh. Johno is much more pragmatic.

OK. Good to know.

Uh. Huh? Is This is true, is it not?

Yes.

OK. So there's you know sort of, yeah, deep, really deep, emotional commitment to a certain theory being uh, complete.

Oh, OK.

You don't have a hidden purist streak?

Oh no.

We - well it hasn't it it certainly hasn't been observed, in any case.

OK. Just checking.

No sir.

Alright.

Um. Now, you do, but that's OK. Uh. So. For for.

Cuz I don't have to implement anything.

Exactly right. Exactly.

I have a problem, then. It's So. Whether I do depends on whether I'm talking to him or him probably.

Hmm.

Yeah, right.

Right. Why a actually, uh, the thing is, you you do but, th the thing you have to im implement is so small that Uh.

Which meeting I'm in. It's OK to be purist within that context.

Within that, yeah,

Yes,

and uh, it's a and still, I think, you know, get something done.

good.

Cool!

But to try to do something upscale and purist Particularly if if um what you're purist about doesn't actually work, is real hard.

Yay.

Mm - hmm. Yeah.

OK. And then the other thing is while we're doing this uh Robert's gonna pick a piece of this space,

It's possible yeah.

OK.

OK, uh, for his absentee thesis. I think you all know that that you can just, in Germany almost just send in your thesis.

Just a drive up. Ca - chuk!

Um

Yeah right.

There you go.

OK.

The - th There there's a drive - in thesis uh sh joint over in Saarbruecken.

Exactly. Drive through, yeah.

It costs a lot. The the amount You put in your credit card and as well. But, uh, But anyway, so, uh, that's um, also gotta be worked out, hopefully over the next few weeks, so that that it becomes clear uh, what piece uh, Robert wants to jump into. And, while we're at this level, uh, there's at least one new doctoral student in computer science who will be joining the project, either next week or the first of August, depending on the blandishments of Microsoft.

OK.

So, de Uh. And her name is Eva.

OK.

It really is. Nobody believed th th that.

Yeah, I thought it had to be a joke, of your part, you know

Yeah.

like  " Johno made it up, I'm sure. "

Is this person someone who's in first - year this year,

No, first year coming.

or

So, she's she's now out here she's moved, and she'll be a student as of then.

OK.

And probably she'll pick up from you on the belief - net stuff, so sh she'll be chasing you down and stuff like that.

OK.

Uh.

Document.

Right.

Uh, against all traditions. And actually I talked today to a uh undergraduate who wants to do an honors thesis on this. Uh.

Someone from the class?

No, interestingly enough.

We always get these people who are not in the class, who.

Some of th some of them, yeah.

It's interesting.

So anyway, uh, but uh she's another one of these ones with a three point nine average and so forth and so on.

Mm - hmm.

Uh, so, um, I've give I've given her some things to read. So we'll see how this goes. Oh there's yet another one of the incoming first  incoming first - year graduate students who's expressed interest, so we'll see how that goes. Um, anyway, so, I think as far as this group goes, um, it's certainly worth continuing for the next few weeks to get closure on the uh belief - net and the ideas that are involved in that, and what are th what are the concepts. We'll see whether it's gonna make sense to have this be separate from the other bigger effort with the formalization stuff or not, I'm not sure. It partly depends on w what your thesis turns out to be and how that goes. S so, we'll see. And then, Ami, you can decide, you know, how much time you wanna put into it and uh, it it's beginning to take shap shape,

OK.

so uh and,

Right

I think you will find that if you want to look technically at some of the your traditional questions in this light, uh Keith, who's buil building constructions, will be quite happy to uh see what, you know, you envision as the issues and the problems and um, how they might uh get reflected in constructions.

Sure.

I suspect that's right.

Yeah. Yeah.

I I may have to go to Switzerland for in June or beginning of July for between two weeks and four weeks, but uh, after that or before that.

OK, fine. And, um, if it's useful we can probably arrange for you to drop by and visit either at Heidelberg or at the German AI center, while you're in in the neighborhood.

Right. Yeah be uh actu actually I'm invited to do some consulting with a bank in Geneva which has an affiliation with a research institute in Geneva, which I forgot the name of.

Yeah. Yep. E o do y

Yeah.

Well, we we're connected to uh There's a there's a a very significant connection between We'll we'll go through this,

Yeah.

ICSI and EPFL, which is the, uh It's the Fr Ge - Germany's got two big technical institutes. There's one in in Zurich,

Mm - hmm.

E T and then there's one, the French speaking one, in Lausanne,

Oh, so in Switzerland.

OK? which is uh E P

Great.

F L. So find out who they are associated with in Geneva.

Right.

Probably we're connected to them.

Great. I'll let you know. S I'll send you email.

OK. Yeah, and so anyway we c uh We can m undoubtedly get Ami uh to give a talk at uh EML or something like that. While he's in in uh.

Hmm. Uh. I I think the one you you gave here a couple of weeks ago would be of interest there, too.

Sure, yeah.

A lot of interest. Actually, either place, DFKI or uh Yeah, so, and and if there is a book, that you'll be building up some audience for it.

Yeah. Right.

And you'll get feedback from these guys.

Great, yeah.

Cuz they've actually these DFKI guys have done as much as anyone over the last decade in trying to build them. So we'll set that up.

Cool.

OK. So, uh, unless we wanna start digging into the uh the belief - net and the decisions now, which would be fine, it's probably.

I I tho It's probably better if I come next week with the um version O point nine of the structure.

OK. So, how about if you two guys between now and next week come up with something that is partially proposal, and partially questions, saying " here's what we think we understand, here are the things we think we don't understand ". And that we as a group will try to to finish it. What I'd like to do is shoot f for finishing all this next Monday.

Sure.

OK? Uh, " these are the decisions " I don't think we're gonna get lots more information. It's a design problem.

Mm - hmm.

You know. We.

Yeah.

Yeah. And let's come up with a first cut at what this should look like. And then finish it up.

OK.

Does that so make sense?

OK.

And um, the the sem semester will be over next week but then you have projects for one more week to come?

No, I I think I'll be done everything by this uh by the end of this week.

Same with you? No.

Nnn. This Well, I've I have projects, but then the my prof professor of one of my classes also wa has a final that he's giving us. And he's giving us five days to do it which means it going to be hard.

Yeah.

Yeah. Oh. is it a take - home final? Who's doing this?

Yeah. Aikin, Alex, yeah.

Yeah, figured. That would have been i my guess.

Hmm.

Right. Um, But anyway, yeah.

Pretty soon.

OK.

OK, so I guess that's

So, the seventeenth will definitely be the last day, like it or not for me.

Right. right. So let's do this, and then we we well there's gonna be some separate co these guys are talking, uh we have a group on the formalization, uh Nancy and Johno and I are gonna talk about parsers. So there're various kinds of uh.

OK.

Of course, nothing gets done even in a meeting of seven people,

Right.

right? So, um, two or three people is the size in which actual work gets done.

Mmm.

Yeah.

So we'll do that. Great. Oh, the other thing we wanna do is catch up with uh, Ellen and see what she's doing because the um image schemas are going to be um, an important pa

Yeah. Quite relevant, yeah.

We we want those,

Yeah, oh yeah.

right? And we want them formalized and stuff like that.

Yeah.

So let me let me make a note to do that.

OK. Yeah, I'm actually probably going to be in contact with her uh pretty soon anyway because of various of us students were going to have a reading group about precisely that sort of thing over the summer,

OK.

Oh right! Right right right!

so.

That's great! Yeah, I I Shweta mentioned that, although she said it's a secret.

OK.

Hi

Right, no faculty!

Th - the faculty aren't faculty aren't supposed to know.

Wednesday's much better for me, yeah.

But um, I'm sufficiently clueless that I count as a.

Yeah, right. It's as if we didn't tell anyone at all,

Bhaskara.

right.

Question: Can you summarize the major points discussed during the meeting?

Response:
* The SmartKom system has been connected to a touch screen, but the system was designed for a different set up.
* The system will be modified to work with the touch screen.
* The lexical entries in the SmartKom grammar will be updated to include the correct tree structures for each lexical item.
* A belief network model will be constructed to manage the inferences needed to answer user queries.
* The belief network model will be designed, implemented, and connected to the rest of the system in the coming weeks.
* The concept-to-speech module will be replaced in October with a better version.
* The system will be demonstrated to the funding agency in October.

Question: What will the researchers do to address the issue of the system being designed for a different setup?

Response:
* The researchers will modify the system to work with the touch screen.
* They will update the lexical entries in the SmartKom grammar to include the correct tree structures for each lexical item.
* They will replace the concept-to-speech module with a better version in October.

Question: They mentioned the system will be connected to a touch screen, but that it was designed for a different set up. What is this set up?

Response: The setup that the SmartKom system was originally designed for is not specified in the meeting transcript.

Question: What was decided regarding the concept-to-speech module?

Response:
* The concept-to-speech module will be replaced with a better version in October.
* The new version will be simpler and less prone to engineering issues.

Question: Can you summarize the major points discussed during the meeting regarding the system's graphical user interface?

Response: I am sorry, but the meeting transcript does not contain any information regarding the system's graphical user interface. Therefore I cannot answer your question.

Question: What did Nancy say regarding the SmartKom dialogue manager in the meeting?

Response: Nancy did not make any comments regarding the SmartKom dialogue manager in the meeting.

Question: Besides the researchers, who else will be involved in modifying the lexical entries in the Smartkom grammar?

Response: I am sorry, but the provided context does not mention who else will be involved in modifying the lexical entries in the Smartkom grammar, besides the researchers. Therefore, I am unable to answer your question.

Question: What did Nancy say regarding the integration of different modules in the SmartKom system in the meeting?

Response: Nancy did not make any comments regarding the integration of different modules in the SmartKom system in the meeting.

Question: What tasks or deliverables were agreed upon at the meeting?

Response:
* The researchers agreed to implement the belief network model in the system in the coming weeks.
* They agreed on the need to update the lexical entries in the Smartkom grammar to include the correct tree structures for each lexical item.
* The researchers will replace the concept-to-speech module with a better version in October.

Question: Did they resolve all the questions raised during the meeting or were there some that were left unanswered?

Response: The transcript does not specify whether all the questions were resolved or not.