Two.

Hello. Hello.

Good morning.

Hello. Ah.

You have to put it exactly on the on the yeah.

Plate?.

Okay.

Good morning.

Good morning.

I took your mouse.

Should I bring my uh pen too?

Yeah just yeah, no,

Or.

that's for me, I just have to make some notes. I got my uh mouse.

Okay.

Uh I also forgot my mouse,

Mouse.

yeah.

but I don't need my mouse, I think.

I do.

Come on. There we are.

My laptop is crashing.

Damn computers..

Cr

Help help help.

Let's just check one more time. Mm.

Can you hear me? Hello?

Uh actually my laptop doesn't work,

Test.

switch it on again.

I dunno.

Oh no.

Check. Okay. I think it works.

Test test. Yes, it's working.

So you all read what we are going to do or not?

Mm-hmm.

Okay.

Yeah.

We're gonna make a remote control.

Yeah, that's right.

I think my laptop is a bit

etchy.

I just made a a simple uh presentation. So you put some things in it.

Okay. How does this work? I dunno..

One uh most to the right.

Uh p

Yes that one.

This one?

Yes.

Press F_ eleven.

Ah cool. Okay. So

that's my name,. Uh we're going to make uh a remote control, you already know that. Just have a look, are we going to uh this agenda of our meeting. You know, this is about twenty five minutes, this meeting. So um the thing we have to know is you already know what we're going to do, you also read what this the things or, not yet,

Yes.

okay. So um, yeah, it has to be original, trendy, user-friendly that's what we're going to design. Uh first we have uh uh three steps of uh making the the remote control. Fir the first thing is th the functional design, that's very important. We have to look what the needs are, the effects of the functional design, and and how the mm the the remote control works, so that's where we're going to look in the functional design, it's for the f next meeting. The the second thing is the conceptual design, that's what it that's uh the spe the specifications of the components and the properties and the specifications of the user interface. And we have to look what uh the market is doing for what kind of uh remote controls are in the market. And the third thing is uh the detailed design um and that's exa yeah, you know what it is, it's exactly how it looks and whatever. Okay so uh no, this is a these are two smartboards, with the uh f uh s an introduction of that one. And you already saw you know all that that you here can put uh things in the the red project uh map.

Folder,

Folder, okay.

yes.

So no okay have a look at that one. Okay. So uh what we're going to do first is um so you can read. You have to uh draw uh a favourite animal on the whiteboard

and um and say why it's your favourite animal. So and you have this is just to try it out and we have to uh use a different pen colours and different thickness of the pen. So okay, so first have to show you, maybe you can come here to have a look how it works. Yes?

Ah I can see it now.

Okay.

Yeah.

This a new page, it's okay. Use pen format. and a different colour can use here no I just take the pink. You take oh there's no pink, okay, oh just purple, okay.

Purple.

No blue. And uh line width ten. Okay uh just take what I'm going to draw is an elephant. Just draw slowly, because otherwise it won't work. It's a very nice elephant, you can see. I dunno what it looks but it doesn't matter..

Looks very nice.

I just h

Something like this? Oh no.

It look like a dinosaurs..

Yeah, okay.

A pink elephant.

Because I like uh okay.

Just takes so long, okay. Whatever, just. You erased this one. It's a bit slow you can see, this is a bit annoying.

Okay.

Okay, so just for you guys just one of you can draw a painting if you want. Just don't

Let me try one.

um yeah, just u use it like that, yeah. That's okay.

Okay.

Mm. Okay and then uh what's the colour? How do I do.

It's in format. Yeah.

Ah. I'll take this one. Uh there has to be water,

Just.

but.

No it has to be an animal, so if that's it's it should be a shna snake or something.

Yeah yeah, but it's an animal it's an animal that lives in the water.

Okay. Okay okay. Okay.

The water is important..

Okay.

So I first uh draw the water.

Uh. Okay, and now I make the animal. It's a fish.

Okay,

Wow.

cool.

Mm-hmm. So. Um.

This is a worm.

Hmm yeah, that's nice.

Wow.

Okay, who next? Uh.

Uh do you have to write down why uh that doesn't matter, just it's to get used to the whiteboard, but it's okay.

Okay.

Just make a new blank new blank page.

Yeah.

Well Paul?.

Yeah. Like this?

Yeah, not too far to the to the t pen top.

Okay. Um

let's make it um a dog. Ooh.

Maybe you should hold your pen a bit more to the back, so that no, to the yeah.

Ah okay.

I think it's a pig.

A pig?

No, it's a dog..

I can make.

Or a dog.

a dog.

Um.

A sheep?.

Uh we d only have twenty five minutes, so..

Take it easy.

'Kay, I make a cat of it.

I I was gonna make a cat too.

Use your fantasy..

Oh, not too quick.

Just hold it more more to the back then you don't have.

No

okay.

I have it. I just draw too quick I think. Okay, that's it. More.

No, that's okay, thank you..

It's just to get used to it.

Okay.

I thought these pens would be just um uh you write it down and you download it to Word, you already did it or no? No, not yet, okay.

No.

Yeah, that's right,

But it's just.

it but you actually got to write on the paper.

Sorry?

You really got to write on that paper.

Yeah, I know, but I d I I thought it would be just in in in uh typed words in Word,

Yeah yeah, it's a real pen.

so that's not

Yeah, but it's just a picture.

just it's just a picture. So.

Oh.

it's not that cool as I th thought it would be.

You really.

Y you can you can't edit in the edit it in Word.

No.

No.

Oh, okay.

It's a donkey.

I don't know, what time did we start this meeting, I'm not sure.

Uh I think it was uh.

Half past.

Half past ten.

Yeah.

Okay.

Brilliant.

Okay, perfect.

Nice, eh?.

Yep.

Yeah, thank you. Now we just have to save everything, so..

Oh

this is definitely the best one.

Uh.

Okay, so uh what we know is that we have to sell this uh remote control for about twenty five Euros. Okay,

Excuse me.

that's. And uh we have to make a profit of fifty million Euros, so we have to uh use a big market in Europe. The production cost are about half the price of selling price,

Piece of cake.

sorry.

Uh easy.

So can someone make a a calculation about how many we have to sell of these to make fifty million, I dunno. Uh so we're gonna have a little discussion about um what experience are with a remote control the and everything, so just have a look how it we think about remote controls.

Yeah, my first question was does it have to be a a universal remote control?

Oh yeah, that's a good question.

Because we sell it uh uh uh lonely from the t and not with the television, we sell it uh apart.

I think it's I'm not I'm not sure,

So I.

it's not mm I think.

Yeah, it probably would be universal.

Universal.

Because.

And only television? Or more devices?

I don't think you can buy a a a universal uh remote control for twenty five Euros, so not sure, but.

Hmm, maybe, I don't know.

Yeah, I know uh you can buy a re a universal uh control for uh only twenty uh Euros

Okay, so we we just say we just say that's universal remote control. Okay, perfect.

Ah okay.

Yeah, I think.

And uh also for the V_C_R_ and uh D_V_D_ player

Yeah, everything

and.

just so a lot of buttons on the remote control.

okay. Yeah, probably.

Not just a T_V_.

No, just everything.

Okay.

. so yeah,

Okay.

what what what's a remote control, it's just a black thing with some buttons on it, it's not nothing very special, but um.

Well we can try to make it special.

yeah, that's right.

So do you have any ideas how it has to look for for usability or user.

Well.

Well I th I thought about um making it the same uh style as the television, we don't have uh the same television uh all the time, so uh that's no matter. Um if we uh control the V_C_R_ and the D_V_D_ player player with it uh it has to be clear, because uh kids and uh elderly are gonna use it, so uh it's not only for the uh technical persons. Um I think it must be a very good control, so you can uh uh uh act uh use it from uh everywhere in your room, the the infrared uh thing must be from very good quality.

Hmm yeah.

That's right. Should be a good point.

Yeah.

Okay.

Nothing N

And how big should it be?

No. Yeah.

It shouldn't be too big,

I dunno um.

but I don't think we can make it too small,'cause it has to have a lot of functions, so.

Yeah.

We can't make it smaller than the things we see now I think.

Just big enough for the buttons we have, that's that's it.

Yeah. Or we have to make something uh like uh mobile phones that you can uh fold it opem.

Yeah. Yeah.

Maybe then you can make it s smaller and uniquer,

Yes

maybe more trendy.

But you you you you think about uh uh one you can fold open.

Fold open,

Okay,

where you can see uh more options. I think.

yeah, that's cool. Maybe for the D_V_D_ pla player or something,

Yeah

Ah

n

that's that's.

if you just.

Yeah, something uh on top, just dren general things like volume and T_V_ channels and inside things you don't use that often.

Ah right right.

Yeah, or you c

Yeah.

O or you could th think of an uh uh a small touch screen on the remote control.

Oh that's good, yeah.

Yeah b I wanted.

Yeah.

Why should you use a touchscreen on a remote control?

Uh maybe be uh it's it's.

There are buttons on it.

No you can make an uh manual in it.

Yeah, or um maybe to keep it uh easy and uh small.

Okay. But it's not t t t too expensive to put a touchscreen on it.

But that's.

Uh like a a to have.

Yeah, but

Yeah, I think it's much uh too expensive.

Yeah, maybe it would.

Yeah but it's not reachable I think, touch screen.

And we have to care for uh the strength of the remote control,

Yeah.

because uh uh remote controls are most uh things in the house that falls down on the ground and it get often uh broken.

Yeah. So. And if you have a touch screen in it, it's definitely too too fragile uh fragile.

Yeah, too fragile.

Yeah and a lots uh uh lots of kids use it and touchscreen is not uh kids uh.

No. You can put games on your remote control. Whatever..

Kid-proof.

Yeah.

And uh how about the batteries?

Um.

Uh should you put it in a recharger or a just.

Oh maybe that's a good idea,

Ma maybe a home station.

just to put it on your television and just s recharge,

Yeah.

you never have to use any batteries.

Mm yeah.

Maybe that's a good idea, but yeah, we have to look at the price now I think.

Yeah, how m how mu how how expensive uh is a normal recharger?

Yeah.

Yeah, I dunno.

Well uh how much is your uh standards uh mobile phone recharger?

I dunno.

Yeah, if you buy it uh separately from your phone it's probably expensive,

Ah yeah.

but I don't know what the project uh projection costs are for such a thing.

Maybe have uh.

Uh if you th look at the market, it's probably it's still the best way just to put batteries in it, because maybe it's too expensive.

Yeah,'cause well.

Yeah.

Yeah, but a home station is uh a really good idea, because uh lots of people are uh lose their remote control and don't know where it is,

Yeah, that's right, yeah.

and now you can put it always at the same place.

Yeah.

Yeah.

Maybe it's Therefore it's a good idea, but maybe it's expensive.

Maybe uh use it as a separate option. Sell it uh separately.

Yeah. You can yeah, you can buy it with it.

Yeah, but I have a mouse that's uh uh also uh.

Rechargeable.

Yeah, and and you just have normal batteries that you can recharge and just put it in the station,

Yeah.

or you can change your batteries,

Oh yeah.

so.

The option, just the option, that's cool.

Yeah.

Uh I I set something on paper already, size, looks, uh usable, uh the buttons on usable places, uh the the on off button must be on top,

Yeah.

uh it has to lay good in the hand,

Yeah.

you you have to uh.

Has it be has does it has to b have to be um uh like a different form than a normal remote control

Yeah, I don't I don't know I don't know

or.

Well I think we have to look at that,

if we.

'cause well you can do the standard way, but then you won't.

You can make it very special, to create our own um looks,

Yeah

but it's very hard to.

N uh if if we want to make it special, we probably have to do a lot of testing, if it really works.

Well you can um have uh the basic things on the same place, like on off button on top and the T_V_ channels one two three four as a block, and then the volume uh obviously on t on top, so you can see.

Nah.

Yeah.

But the rest is uh you don't use that often, so if you have a clear button with an good icon on it, it should be clear as well.

Yeah.

So it doesn't matter what place it is on the remote control I think,

Yeah.

so you can do something unique with that.

And what we have to think about too is uh uh buttons that you use often are um maybe uh like when you fold it open, when you have it closed, you can still uh do the th the functions.

Yeah yeah, what Paul already said.

Yeah, just for the T_V_ and just the normal function, that's fine.

Yeah.

Uh on on top are the the basic options on top,

But n.

Yeah, yeah.

yeah, but uh i basically when I'm watching T_V_ I'm just using like five buttons or so,

and if you fold it open.

But maybe it's

Yeah.

That's what I meant.

so.

maybe it's very hard to to make it fold open and to l look like it's very uh uh what's it's called?

Mm. If you if if you make to fold open it's or also an uh the strength uh is not s as good as a normal uh remote control.

Maybe it's hard t

Uh.

No, that's right. So maybe we have to to uh keep it like mm a square, just normal remote control.

Okay, but yeah.

Just think about it.

Yeah.

We have to close this uh to finish this uh meeting, so.

Okay.

There's some more things. We have uh another thirty minutes, so then we're going to meet again. So you know what you have to what you have to do?

Okay.

Yeah.

Um for m for me uh the Marketing Expert, the user requirements uh specification, do I need to think as a user, a as a a a only the looks and the.

No, what you want to do with it.

Yeah, I think this if you you have to n know if it's for s a T_V_, a D_V_D_ player, all that things. Also from a user, but all these things together.

Okay. Yeah, it's also about strength and uh for everything uh.

Yeah, everything.

Yeah, I also wrote down some stuff

Yeah,

that you want on a.

no maybe not not uh, that's not a f that's something for for for yeah.

Technical fun fu

That's not for you. Just if.

Okay.

what's in the market, what's normal, uh what kind of uh buttons do you have.

What do you want to do with your remote control, what do you need on your remote control.

Yeah, it's alright.

I already wrote some down, some ideas.

Yep.

Yeah, just is that okay?

Yep.

Okay.

Okay.

Okay.

There's already a document in the folder about it.

Yeah, me too.

So see you in thirty minutes.

Okay.

Okay, well done.

Okay.

For the next time you have to uh put it exactly on the square, so your laptop.

I will.

Oh Paul..

It didn't say that.

No, sorry.

Your fault.

Ciao.

Bye bye.

Bye bye.

Question: What did the team meet to discuss?

Response: The team met to discuss the design of a new remote control.

Question: What did the Industrial Designer say about the size of the remote control?

Response: The Industrial Designer did not explicitly mention anything about the size of the remote control.

Question: Besides the size, what other aspects of the remote control design were discussed?

Response:
The participants discussed the following design aspects of the remote control:
* Form factor: should the remote be square shaped or some other form like foldable?
* Strength: the remote should be strong enough to withstand falls.
* Button layout: commonly used buttons should be easily accessible, for example on/off at the top.
* Usability: the remote should be easy to use by everyone, including children and the elderly.

Question: What did the Project Manager say about the color of the remote control?

Response: The color of the remote control was not mentioned during the meeting.

Question: What was the Industrial Designer's opinion on the functionality of the remote control?

Response: The Industrial Designer did not explicitly share their opinion on the functionality of the remote control during the meeting.

Question: Why was it decided that the remote control should be square shaped?

Response: It was not decided that the remote control should be square shaped during the meeting.

Question: Besides the strength and usability, were there any other factors that the team considered when discussing the design of the remote control?

Response:
The participants discussed the following design aspects of the remote control:
-Form factor
-Button layout

Question: What was the advantage of using a touchscreen on the remote control?

Response: The participants discussed on advantage that it will not be expensive to include LCD in the remote control.

Question: Did anyone present an opposing viewpoint when the Marketing Expert suggested having a home station for the remote control?

Response: I apologize but the reference text does not contain any information on an opposing viewpoint being raised about the use of a home station for the remote control.

Question: Based on the items discussed in the meeting, what conclusions can be drawn regarding the target market for this new remote control?

Response: The participants discussed to target the big market in Europe for this new remote control.


What channel am I on?

Channel.

Oh, channel two.

Make sure to turn your microphone on.

Channel.

There's a battery.

There we go.

OK. Your channel number's already on this blank sheet.

Yeah.

So you just If you can.

Channel five? Channel five.

Channel whatever.

I'm on channel five.

Camera one, camera two.

What am I?

Little low?

Channel four?

Channel five.

This number four? OK.

Channel five. OK.

The gai the gain's up at it what it usually is,

Is it?

but if you think it's Yeah. It's sort of a default. But I can set it higher if you like.

Oh. Maybe it should be a little higher.

Yeah?

It's not showing much. Test, test, test, test, test, test, test, test, test, test. OK, that that seems better? Yeah? OK, good. Ah, that's good, that's good. That's Ahh. Mmm. So I I had a question for Adam. Have we started already?

Well, we started recording, but Yeah.

Yeah. Is Jane around or?

I saw her earlier.

Uh.

I think.

She can just walk in, I guess, or.

Yeah. She'll probably come up.

Right.

Since we're starting late I figured we'd better just start.

Yeah. Great idea. I was gonna ask Adam to, uh, say if he thought anymore about the demo stuff because it occurred to me that this is late May and the DARPA meeting is in mid July. Uh, but I don't remember w what we I know that we were gonna do something with the transcriber interface is one thing, but I thought there was a second thing. Anybody remember?

Well, we were gonna do a mock - up, like, question answering or something, I thought, that was totally separate from the interface. Do you remember? Remember, like, asking questions and retrieving, but in a pre - stored fashion.

Mm - hmm. Right.

That was the thing we talked about, I think, before the transcriber.

Yeah.

Come on in.

Alright. So anyway, you have to sort out that out and get somebody going on it cuz we're got a got a month left basically. So.

You like these. Right? OK, good.

OK. Um OK. So, what are we g else we got? You got you just wrote a bunch of stuff.

No. That was all, um, previously here.

Oh.

I was writing the digits and then I realized I could xerox them,

Oh, oh.

because I didn't want people to turn their heads from these microphones. So. We all, by the way, have the same digit form, for the record. So.

That's cool.

Yeah.

So, the choice is, uh, which which do we want more, the the the comparison, uh, of everybody saying them at the same time or the comparison of people saying the same digits at different times that?

It's just cuz I didn't have any more digit sheets.

I know that. But, you know, which opportunity should we

So. Yeah.

Unison.

exploit? Unison.

I mean, it Actually it might be good to have them separately and have the same exact strings. I mean, we could use them for normalizing or something, but it of course goes more quickly doing them in unison.

I guess we'll see

I don't know.

i I guess it's dependent on

See how long we go.

how long we go and how good the snack is out there.

But anyway, they won't be identical as somebody is saying zero in some sometimes, you know, saying O, and so, it's not i not identical.

Yeah. Hmm. Get some advance intelligence.

Right. Right.

Yeah. We'd have to train.

We'd be like a chorus.

OK.

Yeah. We'd have to get s get some experience.

Greek chorus.

Yeah.

Yes.

Yeah. Really boring chorus. Um. Do we have an agenda? Adam usually tries to put those together, but he's ill.

I've got a couple of things to talk about.

So. Yeah. Uh ju what what might those be?

Uh, IBM stuff and, um, just getting uh, meeting information organized.

Meeting info organized. OK. Um.

Are you implying that it's currently disorganized?

In my mind.

Is there stuff that's happened about, um, uh, the SRI recognizer et cetera, tho those things that were happening before with?

Well.

Y y you guys were doing a bunch of experiments with different front - ends and then with Is is that still sort of where it was, uh, the other day?

We're improving.

We're improving.

Yeah.

Now the the You saw the note that the PLP now is getting basically the same as the MFCC.

Right.

Right?

Yeah. Actually it looks like it's getting better.

Right. Oh.

So. But but it's not.

Just with with age, kind of.

With age. Yeah.

Yeah. Yeah.

But, uh, that's not d directly related to me. Doesn't mean we can't talk about it. Um, it seems It looks l I haven't The It's The experiment is still not complete, but, um, it looks like the vocal tract length normalization is working beautifully, actually, w using the warp factors that we computed for the SRI system and just applying them to the ICSI front - end.

Mm - hmm. That's pretty funny.

Yeah.

OK.

So you just need to copy over to this one.

Just had to take the reciprocal of the number because they have different meanings in the two systems.

OK.

Ah! Yeah. Well, that's always good to do.

Yeah.

OK. OK. Uh.

But one issue actually that just came up in discussion with Liz and and Don was, um, as far as meeting recognition is concerned, um, we would really like to, uh, move, uh, to, uh, doing the recognition on automatic segmentations.

Yeah.

Because in all our previous experiments, we had the uh, you know, we were essentially cheating by having the, um, you know, the h the hand - segmentations as the basis of the recognition.

Mm - hmm.

And so now with Thilo's segmenter working so well, I think we should consider doing a.

Mmm. So.

Come on.

uh, doing.

Yeah. We But.

Y think you think we should increase the error rate.

Anyway. Yeah.

Yeah.

Yeah.

Good.

Yeah.

Yeah.

That - that's what I wanted to do anyway,

Yeah.

so we should just get together and.

Yeah.

Yeah.

And even The good thing is that since you, um, have high recall,  even if you have low precision cuz you're over - generating, that's good because we could train noise models in the recognizer for these kinds of, uh, transients and things that come from the microphones,

Right.

Yeah. Yeah.

but I know that if we run recognition unconstrained on a whole waveform, we do very poorly because we're we're getting insertions in places what that you may well be cutting out.

Well.

Yeah.

Mm - hmm.

So we do need some kind of pre - segmentation.

We should we should consider doing some extra things, like, um, you know, retraining or adapting the the models for background noise to the to this environment, for instance.

Mmm. Yeah.

Yeah.

And, yeah, using Thilo's, you know, posteriors or some kind of or.

So.

right now they're they're discrete,

Yeah.

yes or no for a speaker, to consider those particular speaker background models.

Right.

So. There's lots of ins interesting things that could be done.

Yeah. Yeah. We should do that.

So.

Good. So, uh, why don't we, uh, do the IBM stuff?

Yeah. So, um, talked with Brian and gave him the alternatives to the single beep at the end of each utterance that we had generated before.

You had some thing about that? Right.

And so.

The, uh, Chuck chunks.

Yeah. The Chuck chunks.

Hmm.

Right. And so he talked it over with the transcriber and the transcriber thought that the easiest thing for them would be if there was a beep and then the nu a number, a digit, and then a beep, uh, at the beginning of each one

Yeah. Yeah.

and that would help keep them from getting lost. And, um, so Adam wrote a little script to generate those style, uh, beeps

Where'd you get the digits from?

and so we're I came up here and just recorded the numbers one through ten.

They sound really good.

So. Does it sound OK?

That's a great idea.

Yeah.

So, um Yeah. We just used those.

And do you splice them into the waveform? Or?

Yeah. He then he d I recorded Actually, I recorded one through ten three times at three different speeds and then he picked.

Right. Mm - hmm.

He liked the fastest one, so he just cut those out and spliced them in between, uh, two beeps.

It sounds like a radio announcer's voice. Really.

It will be funny uh.

Yeah, yeah.

Does it?

It will be funny when you're really reading digits, and then there are the chunks with with your digits in?

Yeah. With my.

Oh that's right.

Oh, right.

Yeah.

Now actually,

That'll throw them,

we're Are we handling?

huh?

Uh, maybe we should have you record A, B, C for those or something.

Yeah.  Huh! Maybe. And she said it wasn't gonna the transcriber said it wouldn't be a problem cuz they can actually make a template, uh, that has beep, number, beep. So for them it'll be very quick

OK.

to to put those in there when they're transcribing.

Yeah.

So, um, we We're gonna send them one more sample meeting, uh, and Thilo has run his segmentation. Adam's gonna generate the chunked file. And then, um, I'll give it to Brian and they can try that out. And when we get that back we'll see if that sort of fixes the problem we had with, uh, too many beeps in the last transcription.

OK. Do w do what Do you have any idea of the turn - around on on those steps you just said?

Great.

Uh. Our s our On our side?

Uh.

or including IBM's?

Including IBM's.

Well, I don't know. The last one seemed like it took a couple of weeks. Um, maybe even three.

OK.

Uh, that's just the I B M side. Our side is quick. I mean, I I don't know. How long does your?

It should @ @ be finished today or something. Yeah.

Well, I meant the overall thing.

Yeah.

e e u u  The reason I'm asking is because, uh, Jane and I have just been talking, and she's just been doing. Uh, e a, you know, further hiring of transcribers.

Mm - hmm. Mm - hmm.

And so we don't sort of really know exactly what they'll be doing, how long they'll be doing it, and so forth, because right now she has no choice but to operate in the mode that we already have working.

Right.

And, uh, so it'd be It'd be good to sort of get that resolved, uh, soon as we could,

Yeah. I Yeah, I I hope @ @  we can get a better estimate from this one that we send them.

and then.

So. Um.

Mm - hmm.

I I don't know yet how long that'll take.

Yeah. Um I mean in particular I would I would really hope that when we do this DARPA meeting in July that we sort of have we're we're into production mode, somehow.

Mm - hmm.

You know, that we we actually have a stream going and we know how how well it does and how and how it operates.

Yeah.

I think that would that would certainly be a a very good thing to know.

Right. Right.

OK. Uh. Maybe before we do the meeting info organize thing, maybe you could say relevant stuff about where we are in transcriptions.

OK. So, um, we Uh, the transcribers have continued to work past what I'm calling " set one ", which was the s the set that I've been, uh OK, talking about up to this point, but, uh, they've gotten five meetings done in that set. Right now they're in the process of being edited. Um, the, um Let's see, I hired two transcribers today. I'm thinking of hiring another one, which will because we've had a lot of attrition. And that will bring our total to.

They die off after they do this for a while.

Yeah. Well, you know, it's it's various things.

Burn - out.

Yeah.

So, one of them had a baby. Um, you know, one of them really w wasn't planning.

Oh, that was an unfor unforeseen side effect of.

Eh, one of them, um, had never planned to work past January. I mean, it's th all these various things, cuz we, you know, we presented it as possibly a month project back in January and and and and Um, so it makes sense. Uh, through attrition we we've we're down to to two, but they're really solid. We're really lucky the two that we kept. And, um Well, I don't mean I don't mean anything against the others.  What I mean is we've got a good cause a good core. No. We had a good core.

Well, they won't hear this since they're going. They won't be transcribing this meeting.

Yeah, but still. I mean, I d it's just a matter of we w we're we've got, uh,

No backs.

two of the ones who who, um, ha had been putting in a lot of hours up to this point and they're continuing to put in a a lot of hours, which is wonderful, and excellent work. And so, then, in addition, um, I hired two more today and I'm planning to h hire a third one with this within this coming week, but but the plan is just as, uh, Morgan was saying we discussed this, and the plan right now is to keep the staff on the on the leaner side, you know, rather than hiring, like, eight to ten right now,

Mm - hmm.

because if the IBM thing comes through really quickly, then, um, we wouldn't wanna have to, uh, you know, lay people off and stuff. So. And this way it'll I mean, I got really a lot of response for for my notice and I think I could hire additional people if I wish to.

Yeah. An - and the other thing is, I mean, in the unlikely event and since we're so far from this, it's a little hard to plan this way in the unlikely event that we actually find that we have, uh, transcribers on staff who are twiddling their thumbs because, you know, there's, you know, all the stuff that that was sitting there has been transcribed and they're and they're faster the the pipeline is faster than uh, than the generation, um, eh, i in in the day e event that that day actually dawns, uh, I I bet we could find some other stuff for them to do.

Oh, yes.

So I I think that, eh, eh, a as we were talking, if we if we hire twelve, then we could, you know, run into a problem later. I mean, we also just couldn't sustain that forever. But but, um for all sorts of reasons but if we hire f you know, f we have five on staff five or six on staff at any given time, then it's a small enough number so we can be flexible either way.

Good. OK.

Good.

It'd be great, too, if, um, we can we might need some help again getting the tighter boundaries or some hand to experiment with, um you know, to have a ground truth for this segmentation work, which I guess you have some already that was really helpful, and we could probably use more.

Mmm, yeah. That was a thing I I planned working on, is, uh, to use the the transcriptions which are done by now, and to to use them as, uh.

Oh. Oh, the new ones

Yeah.

with the tighter boundaries. Yeah.

Yeah. And to use them for for training a or for fo whatever. Yeah. To to create some speech - nonspeech labels out of them, and Yeah, but that that's a thing w was w what I'm just looking into.

OK.

The the the pre - segmentations are so much are s so extremely helpful. Now there was, uh, I g guess So, a couple weeks ago I needed some new ones and it happened to be during the time that he was on vacation f for just very few days you were away. But it happened to be during that time I needed one,

Yeah.

so I so I started them on the non - pre - segmented and then switched them over to yours and, um, they, um you know, they always appreciate that when they have that available. And he's, uh, usually, eh, uh, um Um. So they really appreciate it. But I was gonna say that they do adjust it once in a while. You know, once in a while there's something like,

Yeah, sure.

um, and e Actually you talked to them. Didn't you? Did you? Have you?

Yeah. I talked to Helen.

And and and she was And so, I asked her I mean, They're very perceptive. I really want to have this meeting of the transcribers. I haven't done it yet, but I wanna do that and she's out of town, um, for a couple of weeks, but I wanna do that when she returns. Um, cuz she was saying, you know, in a in a span of very short period we asked It seems like the ones that need to be adjusted are these these these things, and she was saying the short utterances, uh, the, um.

Hmm.

Mmm. Yeah.

you know, I mean, you're You're aware of this. But but actually i it's so correct for so much of the time, that it's an enormous time saver

Yeah.

and it just gets tweaked a little around the boundaries. So.

That's great.

Um. Yeah. I think it'd be interesting to combine these.

Yeah.

Is there actually a record of where they change? I mean, you can compare, do a diff on the just so that we knew.

You could do it. It's it's complicated in that um, hhh, i hhh, i

Yeah. Actually, when when they create new yeah, new segments or something, it will be, uh, not that easy but hmm. I think one could do that.

I mean, if we keep a old copy of the old time marks

Yeah.

just so that if we run it we know whether we're which ones were cheating

Yeah. Yeah. That would be great, yeah, to know that.

and

There is a there is one problem with that and that is when they start part way through then what I do is I merge what they've done with the pre - segmented version.

which one would be good.

Yeah.

So it's not a pure it's not a pure condition. Wha - what you'd really like is that they started with pre - segmented and were pre - segmented all the way through.

Mm - hmm.

And, um @ @  I, uh the it wasn't possible for about four of the recent ones. But, it will be possible in the future

Yeah.

because we we're, um.

It would.

Mmm, that's great.

Yeah.

Yeah. As long as we have a record, I guess, of the original automatic one, we can always find out how well we would do fr from the recognition side by using those boundaries.

Yeah. Yeah.

Um.

Yeah.

You know, a completely non - cheating version.

Yeah.

Also if you need someone to record this meeting, I mean, I'm happy to for the transcribers I could do it, or Chuck or Adam.

Thank you.

OK. So, uh, u you were saying something about organizing the meeting info?

Yeah. So, um, uh, Jane and Adam and I had a meeting where we talked about the reorganization of the directory structure for all of the meeting.

Did you record it?

No. For all the Meeting Recorder data. We should have. Um. And so we've got a plan for what we're gonna do there. And then, Jane also s prepared a um, started getting all of the the meetings organized, so she prepared a a spreadsheet, which I spent the last couple of days adding to. So I went through all of the data that we have collected so far, and have been putting it into, uh, a spreadsheet with start time, the date, the old meeting name, the new meeting name, the number of speakers, the duration of the meeting, comments, you know, what its transcription status is, all that kind of stuff. And so, the idea is that we can take this and then export it as HTML and put it on the Meeting Recorder web page so we can keep people updated about what's going on.

Oh, great.

Um, I've gotta get some more information from Jane cuz I have some some gaps here that I need to get her to fill in, but so far, um, as of Monday, the fourteenth, um, we've had a total number of meeting sixty - two hours of meetings that we have collected. And, um Uh, some other interesting things, average number of speakers per meeting is six. Um, and I'm gonna have on here the total amount that's been transcribed so far, but I've got a bunch of uh, that's what I have to talk to Jane about, figuring out exactly which ones have have been completed and so forth. But, um, this'll be a nice thing that we can put up on the the web site and people can be informed of the status of various different ones. And it'll also list, uh, like under the status, if it's at IBM or if it's at ICSI, uh, or if it's completed or which ones we're excluding and and there's a place for comments, so we can, um, say why we're excluding things and so forth. So.

Now would the ones that, um, are already transcribed we h we have enough there that c you know, we've already done some studies and so forth and um, shouldn't we go through and do the business - es u of of having the, um, uh, participants approve it, uh, for approve the transcriptions for distribution and so forth?

Um, interesting idea. In principle, I I would say yes, although I still am doing some the final - pass editing, trying to convert it over to the master file as the being the channelized version and it's Yeah, it seems like I get into that a certain way and then something else intervenes  and I have to stop. Cleaning up the things like the, uh, uh, places where the transcriber was uncertain, and and doing spot - checking here and there. So, um, uh, I guess it would make sense to wait until th that's done, um, but but.

Well, le let me put in another sort of a milestone kind of as as I did with the, uh, uh the the pipeline.

Yeah.

Um, we are gonna have this DARPA meeting in the middle of July,

Yes.

and I think it w it'd be given that we've been we've given a couple public talks about it already, spaced by months and months, I think it'd be pretty bad if we continued to say none of this is available. Um.

It'll certainly be done by then. Yeah.

Right. So we can s we we wanna be able to say " here is a subset that is available right now "

Mm - hmm. That's right.

and that's has been through the legal issues and so forth.

That's right.

So.

Yeah. That's right. So that.

OK?

OK.

So, by before July.

And they don't have to approve, you know, th an edited version, they can just give their approval to whatever version

Well, maybe.

Well, in principle, yes. But, I mean, i if if if somebody actually did get into some legal issue with it then we

Bu Yeah. But th I mean, the editing will continue. Presumably if if s errors are found, they will be fixed, but they won't change the the content of the meetings.

Content, really.

Well, see, this is the this is the issue. Subtleties.

So.

Well, i if Jane is clarifying question question, then, you know, how can they agree to it before they know her final version?

The other thing, too, is there can be subtleties where a person uses this word instead of that word, which @ @  could've been transcribed in the other way.

Yeah.

Thing.

And no and they wouldn't have been slanderous if it had been this other word. You know?

I it you know, there there is a point at which I agree it becomes ridiculous because, you know, you could do this final thing and then a year from now somebody could say, you know, that should be a period and not a question mark. Right? And you don't you there's no way that we're gonna go back and ask everybody " do you approve this, uh, you know this document now? " So So I think what it is is that the the the the thing that they sign I I haven't looked at it in a while, but it has to be open enough that it sort of says " OK, from now on you know, now that I've read this, you can use do anything you want with these data. "

Mm - hmm.

And, uh But, i I think we wanna So, assuming that it's in that kind of wording, which I don't remember, um, I think i we just wanna have enough confidence ourselves that it's so close to the final form it's gonna be in, a year from now that they're.

Mm - hmm. I agree. Mmm. I totally agree. It's just, uh, a question of, uh, if if the person is using the transcript as the way of them judging what they said and whether it was slanderous, then it seems like it's it's i it needs to be more correct than if we could count on them re - listening to the meeting.

Uh.

Because it becomes, eh, in a way a a f uh, a legal document i if they've agreed to that.

Well, I forget how we end Right. I forget how we ended up on this, but I remember my taking the position of not making it so so easy for everybody to observe everything and Adam was taking the position of of having it be really straightforward for people to check every aspect of it including the audio. And I don't remember who won, Adam or me, but.

Well, if it's only the transcript, though I mean, th this this is my point, that that

uh, the Uh, that that's why I'm bringing this up again, because I can't remember how we ended up.

then it becomes.

That it was the transcrip He wanted to do a web interface that would make it.

Well, if it's just the audio Well.

that would give you access to the transcript and the audio. That's what Adam wanted.

Mm - hmm.

And I don't remember how we ended up.

I mean, with the web interface it's interesting, because you could allow the person who signs to be informed when their transcript changes, or something like that. And, I mean, I would say " no ". Like, I don't wanna know, but some people might be really interested and then y In other words, they would be informed if there was some significant change other than typos and things like that.

You decided you were whispering Satanic incantations under your breath when you were.

Well, I don't know what happened to the small heads thing, but I j Um, I'm just saying that, like, you know, you can sort of say that any things that are deemed.

They disappeared from view.

Anyway. I mean, I agree that at some point people probably won't care about typos but they would care about significant meaning changes and then they could be asked for their consent, I guess, if if those change. Cuz assumi assuming we we don't really distribute things that have any significant changes from what they sign anyway.

Tha That's How about having them approve the audio and not the transcripts?

Oh, my God.

That would be simpler,

Uh.

if we could count on them listening.

But no one will listen to the hours and hours of.

Talk.

Well, that's O K.

That's.

We just have to give them a chance to listen to it, and if they don't, that's their problem.

hmm, hmm.

You you d That's like.

Unfortunately, uh, in in the sign thing that they signed, it says " transcripts ".

No, I'm serious.

" You'll be you'll be provided the transcripts when they're available. "

Really?

Mmm.

Mmm.

Yeah.

Yeah.

I I I think

Yeah.

that's a lot to ask for people that have been in a lot of meetings.

Yeah.

W anyway, haven't we we've gone down this path a number of times. I know this can lead to extended conversations and and not really get anywhere, so let let me just suggest that uh, off - line that, uh, the people involved figure it out and take care of it before it's July.

Yes.

OK. So so that in July we can tell people " yes, we have this and you can use it ".

Yes. It's done, ready, available. Good.

Uh. So, let's see. What else we got? Uh. Don did did a report about his project in class and, uh an oral and written written version.

Well.

So that was stuff he was doing with you. Yeah.

I mean, it's I guess one thing we're learning is that the amount We have eight meetings there because we couldn't use the non - native all non - native meetings and it's, well, probably below threshold on enough data for us for the things we're looking at because the prosodic features are very noisy and so you you need a lot of data in order to model them. Um, so we're starting to see some patterns and we're hoping that maybe with, I don't know, double or triple the data with twenty meetings or so, that we would start to get better results. But we did find that some of the features that, I gue Jane would know about, that are expressing sort of the distance of, um, boundaries from peaks in the utterance and some local, um, range pitch range effects, like how close people are to their floor, are showing up in these classifiers, which are also being given some word features that are cheating, cuz they're true words. Um, so these are based on forced alignment. Word features like, um, word frequency and whether or not something's a backchannel and so forth. So, we're starting to see, I think, some interesting patterns.

So the dominant features, including everything, were those those quasi - cheating things. Right? Where these are.

Sometimes not.

I think it depends what you're looking at, a actually.

Yeah. Sometimes positions in sentences obviously, or in spurts, was helpful. I don't know if that's cheating, too.

Right. Um,

Spurts wouldn't be. Right?

spurts is not cheating except that of course you know the real words,

Right.

but roughly speaking, the recognized words are gonna give you a similar type of position.

Right. Would they give you the same number of words, though?

Right.

It's either early or late.

No

Not exactly, but i

But ra somewhat?

On the average.

Y yeah it should be. Well, we don't know and actually that's one of the things we're interested in doing, is a sort of.

Uh - huh.

Have you tried using just time, as opposed to number of words?

So.

I think ti uh Just p time position, like when the word starts?

Yeah.

I don't know if that was in the.

Well, no, I mean t time time position relative to the beginning of the spurt.

Eh You know, uh

Start.

Yeah,

Yeah. There's all these things to do.

uh, we didn't try it, but it's s

Like, there's a lot of different features you could just pull out.

Yeah. I mean that wouldn't be cheating because you can detect pause pretty well within the time.

Right.

Right.

How about time position normalized by speak

And it depends on speaking rate.

Yeah. Yeah.

speaking rate. Yeah.

Yeah.

Yeah. That's actually why I didn't use it at first.

Yeah.

Mm - hmm.

But we one of the interesting things was I guess you reported on some te punctuation type.

Yeah.

finding sentence boundaries, finding disfluency boundaries, and then I had done some work on finding from the foreground speech whether or not someone was likely to interrupt, so where you know, if I'm talking now and someone and and Andreas is about to interrupt me, is he gonna choose a certain place in my speech, either prosodically or word - based. And there the prosodic features actually showed up and a neat thing even though the word features were available. And a neat thing there too is I tried some putting the speaker So, I gave everybody a short version of their name. So the real names are in there, which we couldn't use. Uh, we should use I Ds or something. And those don't show up. So that means that overall, um, it wasn't just modeling Morgan, or it wasn't just modeling a single person,

Mm - hmm.

um, but was sort of trying to, uh, get a general idea the model the tree classifier was trying to find general locations that were applicable to different speakers, even though there are huge speaker effects. So. The but the main limitation now is I because we're only looking at things that happen every ten words or every twenty words, we need more more data and more data per speaker. So. It'd also be interesting to look at the EDU meetings because we did include meeting type as a feature, so whether you were in a r Meeting Recorder meeting or a Robustness meeting did matter to interrupts because there are just fewer interrupts in the Robustness meetings.

Mm - hmm.

And so the classifier learns more about Morgan than it does about sort of the average person,

Mm - hmm.

which is not bad. It'd probably do better than Um, but it wasn't generalizing.

Yeah.

So it's And I think Don, um Well, we have a long list of things he's starting to look at now over the summer, where we can And he'll be able to report on more things in the future. But it was great that we could at least go from the you know, Jane's transcripts and the, uh, recognizer output and get it to this point. And I think it's something Mari can probably use in her preliminary report like, " yeah, we're at the point where we're training these classifiers and we're just reporting very preliminary but suggestive results that some features, both word and pro prosodic, work. " The other thing that was interesting to me is that the pitch features are better than in Switchboard. And I think that really is from the close - talking mikes, cuz the pitch processing that was done has much cleaner behavior than than the Switchboard telephone bandwidth.

W wh wh wh Better in what sense?

Um. Well, first of all, the pitch tracks are m have less, um, halvings and doublings than than Switchboard and there's a lot less dropout, so if you ask how many regions where you would normally expect some vowels to be occurring are completely devoid of pitch information,

Mm - hmm..

in other words the pitch tracker just didn't get a high enough probability of voicing for words for for, you know, five word

Hmm.

there are much fewer than in Switchboard. So the missing We had a big missing data problem in Switchboard and, so the features weren't as reliable cuz they were often just not available.

Could it have to do with the the lower frequency cut - off on the Switchboard?

So that's actually good. Ma - maybe. I mean, the tele we had telephone bandwidth for Switchboard and we had the an annoying sort of telephone handset movement problem that I think may also affect it.

Hmm.

So we're just getting better signals in in this data. Which is nice. So.

Yeah.

Anyway, Don's been doing a great job and we hope to continue with, um, Andreas's help and also some of Thilo's help on this,

Great.

Y

to to try to get a non - cheating version of how all this would work.

Yeah. Sure. Yeah.

Has has, uh? We just I think, just talked about this the other day, but h has has anybody had a chance to try changing, uh, insertion penalty sort of things with the with the, uh uh, using the tandem system input for the?

Oh, yeah. I tried that. It didn't, um, help dramatically. The.

Were they out of balance? I didn't I didn't notice.

There were a little the relative number of I think there were a higher number of deletions, actually.

Oh.

So, you, uh So, actually it it preferred to have a positive er, negative insertion penalty,

Deletions?

which means that, um.

Uh - huh.

But, you know, it didn't change th the by adjusting that the, um.

OK.

Yeah. The error changed by probably one percent or so. But, you know, given that that word error rate is so high, that's not a.

OK. So that So that's So that's not the problem.

That's not the problem. No.

Yeah.

But, uh, we s just, um, uh you know, Chuck and I talked and the @ @  next thing to do is probably to tune the um, the size of the Gaussian system, um, @ @  to to this to this feature vector, which we haven't done at all. We just used the same configuration as we used for the for the standard system.

Hmm.

And, for instance, uh, Dan @ @  Dan just sent me a message saying that CMU used, um, something like ten Gaussians per cluster You know, each each mixture has ten Gaussians

Mm - hmm. Hmm. We're using sixty - four,

and and we're using sixty - four,

right?

so that's obviously a big difference

Yeah.

and it might be way off and give very poorly trained, uh, you know, Gaussians that way,

Hmm.

uh, an and poorly trained mixture weights. So so, we have The turn - around time on the training when we train only the a male system with, uh, you know, our small training set, is less than twenty - four hours, so we can run lots of uh, basically just brute force, try a whole bunch of different um, settings.

OK.

And, uh, with the new machines it'll be even better. So.

Yeah. We get twelve of those,

Yeah.

huh?

But the PLP features work um, uh, you know, continue to improve the,

OK.

um As I said before, the uh using Dan's, uh, uh, vocal tract normalization option works very well. So, um, @ @  I ran one experiment where we're just did the vocal tract le normalization only in the test data,

Mm - hmm.

so I didn't bother to retrain the models at all, and it improved by one percent, which is about what we get with uh, with, you know, just @ @  actually doing both training and test normalization, um, with, um, the, uh uh, with the standard system. So, in a few hours we'll have the numbers for the for retraining everything with vocal tract length normalization and So, that might even improve it further.

Great.

So, it looks like the P L - fea P features  do very well now with after having figured out all these little tricks to to get it to work.

Yeah.

So.

Good.

Wait. So you mean you improve one percent over a system that doesn't have any V T L in it already?

Exactly. Yeah.

OK.

Yeah. OK. So then then we'll have our baseline to to compare the currently hideous, uh, uh, new thing with.

Right. a Right. And and what that suggests also is of course that the current Switchboard MLP isn't trained on very good features.

But Yeah.

Uh, because it was trained on whatever, you know, was used, uh, last time you did Hub - five stuff, which didn't have any of the.

Right. But all of these effects were j like a couple percent.

Uh.

Right? I mean, y the.

Well, but if you add them all up you have, uh, almost five percent difference now.

Add all of them. I thought one was one point five percent and one was point eight.

Yeah. And now we have another percent with the V T

That's three point three.

Um, actually, and it's,  um, What's actually qu interesting is that with um, well, you m prob maybe another half percent if you do the VTL in training, and then interestingly, if you optimize you get more of a win out of rescoring the, um, uh, the N best lists, uh, and optimizing the weights, um, uh than.

Than you do with the standard?

Yeah. So.

Yeah. But the part that's actually adjustment of the front - end per se as opposed to doing putting VTLN in or something is it was a couple percent.

Right.

Right? It was it was there was there was one thing that was one and a half percent and one that was point eight. So and and let me see if I remember what they were. One of them was, uh, the change to, uh because it did it all at once,  to uh, from bark scale to mel scale,

Mm - hmm.

which I really feel like saying in quotes, because @ @  they're essentially the same scale but the but but but any i individual particular implementation of those things puts things in a particular place.

Yeah. Why did that cha?

Mm - hmm.

So that's why I wanted to look I still haven't looked at it yet. I I wanna look at exactly where the filters were in the two,

Mm - hmm.

and it it's probably something like there's one fewer or one more filter in the sub one kilohertz band

Mm - hmm.

and for whatever reason with this particular experiment it was better one way or the other.

Hmm.

Um, it could be there's something more fundamental but it you know, I I don't know it yet. And the other and the other that was like one and a half or something, and then there was point eight percent, which was what was the other thing?

Well, that was combined with the triangular. Right?

Yeah. Those those two were together.

Yeah. Right.

We d weren't able to separate them out cuz it was just done in one thing. But then there was a point eight percent which was something else.

The low - frequency cut - off.

Do you remember the? Oh, yeah. So that was that was, uh that one I can claim credit for, uh, i in terms of screwing it up in the first place. So that someone e until someone else fixed it, which is that, um, I never put when I u We had some problems before with offsets. This inf this went back to, uh, I think Wall Street Journal.

Hmm.

So we we had, uh ea everybody else who was doing Wall Street Journal knew that there were big DC offsets in th in these data in those data and and and nobody happened to mention it to us,

Hmm.

and we were getting these, like, really terrible results, like two, three times the error everybody else was getting. And then in casual conversation someone ment mentioned " uh, well, I guess, you know, of course you're taking care of the offsets. " I said " what offsets? "

Mm - hmm.

And at that point, you know, we were pretty new to the data and we'd never really, like, looked at it on a screen and then when we just put it on the screen  and wroop!

Mm - hmm.

there's this big DC offset. So, um, in PLP

There was a like a hum or some or when they recorded it?

No. It's just, it it's it's not at all uncommon for for recorded electronics to have different, um, DC offsets.

Or just? Huh.

It's it's, you know, no big deal. It's you know, you could have ten, twenty, maybe thirty millivolts, whatever, and it's consistently in there. The thing is, most people's front - ends have pre - emphasis with it, with zero at zero frequency, so that it's irrelevant. Uh, but with P L P, we didn't actually have that. We had we had the equivalent of pre - emphasis in a a, uh, Fletcher - Munson style weighting that occurs in the middle of P L but it doesn't actually have a zero at zero frequency,

Hmm.

like, eh, uh, typical simple fr pre - emphasis does. We had something more fancy. It was later on it didn't have that. So at that point I reali " oh sh we better have a have a high - pass filter " just, you know just take care of the problem. So I put in a high - pass filter at, uh, I think ninety ninety hertz or so uh, for a sixteen kilohertz sampling rate. And I never put anything in to adjust it for different different sampling rates. And so well, so, you know, the code doesn't know anything about that and so this is all at eight kilohertz and so it was at forty - five hertz instead of at instead of at ninety.

Hmm.

So, um, I don't know if Dan fixed it or or, uh, what he.

Well, he made it a parameter.

He made it a parameter. So. Yeah, I guess if he did it right, he did fix it and then and then it's taking care of sampling rate, which is great.

What what is the parameter?

He had a.

Is it, uh, just the f lower cut - off that you want?

It's called, uh, H - HPF.

H Yeah. Does HPF on on his feat feature.

u And but HPF, you know, when you put a number after it, uses that as the hertz value of the cut - off.

Mm - hmm. Oh, OK.

Yeah.

So.

I mean, frankly, we never did that with the RASTA filter either,

Mm - hmm.

so the RASTA filter is actually doing a different thing in the modulation spectral domain depending on what sampling rate you're doing, which is another old old bug of mine.

Mm - hmm.

But, um Um. So that that was the problem there was th we we we had always intended to cut off below a hundred hertz

Mm - hmm.

and it just wasn't doing it, so now it is. So, that hep that helped us by, like, eight tenths of a percent. It still wasn't a big deal.

OK. Well, but, um Well, uh, again, after completing the current experiments, we'll we can add up all the uh differences

Oh, yeah.

and and an

But but, I guess my my point was that that, um, the hybrid system thing that we did was, uh, primitive in many ways.

Y Right.

And I think I agree with you that if we fixed lots of different things and they would all add up, we would probably have a a a competitive system. But I think not that much of it is due to the front - end per se. I think maybe a couple percent of it is, as far as I can see from this.

Mm - hmm.

Uh, unless you call well, if you call VTL the front - en front - end, that's, uh, a little more. But that's sort of more both, kind of.

One experiment we should we'll probably need to do though when um, at some point, is, since we're using that same the net that was trained on PLP without all these things in it, for the tandem system, we may wanna go back and retrain,

Right? But.

Well, that's what I meant, in fact. Yeah.

yeah, yeah, for the tandem. You know, so we can see if it what effect it has on the tandem processing.

So so, the thing is is do we expect?

Mm - hmm.

eh At this point I'm as I mean, you know e I'm wondering is it Can we expect, uh, a tandem system to do better than a properly trained you know, a Gaussian system trained directly on the features with, you know, the right ch choice of parameters?

Well, that's what we're seeing in other areas. Yes. Right? So, it's so, um, um.

So, we But but we may not. I mean, if it doesn't perform as well, we may not know why. Right? Cuz we need to do the exact experiment.

Right.

I mean, the reason to think it should is because you're putting in the same information and you're transforming it to be more discriminative. So. Um. Now the thing is, in some databases I wouldn't expect it to necessarily give you much and and part of what I view as the real power of it is that it gives you a transformational capability for taking all sorts of different wild things that we do, not just th the standard front - end, but other things, like with multiple streams and so forth,

Mm - hmm.

and allows you to feed them to the other system with this through this funnel. Um, so I think I think that's the real power of it. I wouldn't expect huge in huge improvements. Um, but it should at least be roughly the same and maybe a little better.

Mm - hmm.

If it's, you know, like way way worse then, you know.

Right.

So, Morgan, an another thing that Andreas and I were talking about was, so @ @  in the first experiment that he did we just took the whole fifty - six, uh, outputs and that's, um, basically compared to a thirty - nine input feature vector from either MFCC or PLP.

Mm - hmm. Mm - hmm.

But one thing we could do is.

Let let me let me just ask you something. When you say take the fifty - six outputs, these are the pre final nonlinearity outputs

Yeah. Through the regular tandem outputs.

and they're and through the KLT.

Through the KLT. All that kinda stuff.

OK. And so so then you u Do you use all fifty - six of the KLT

That's what we did.

or?

Right? So one thing we were wondering is, if we did principal components and, say, took out just thirteen, and then did deltas and double - deltas on that.

OK. Yes.

so we treated the th first thirteen as though they were standard features.

Yes. Yeah.

I mean, did Dan do experiments like that to?

Uh. Talk with Stephane. He did some things like that. It was either him or Carmen. I forget.

Mm - hmm.

Mmm.

I mean these were all different databases and different you know, in HTK and all that,

Yeah.

so i it it may not apply. But my recollection of it was that it didn't make it better but it didn't make it worse.

Hmm.

But, again, given all these differences, maybe it's more important in your case that you not take a lot of these low - variance, uh, components.

Cuz in a sense, the net's already got quite a bit of context in those features,

Yeah.

so if we did deltas and double - deltas on top of those, we're getting sort of even more.

Which could be good or not.

Yeah.

Yeah. Yeah. Worth trying.

But there the main point is that, um, you know, it took us a while but we have the procedure for coupling the two systems debugged now and I mean, there's still conceivably some bug somewhere in the way we're feeding the tandem features uh, either generating them or feeding them to this to the SRI system,

Mm - hmm. Yeah.

but it's.

There might be, cuz that's a pretty big difference.

Yeah. And I'm wondering how we can how we can debug that.

But

Yeah.

I mean how Um.

Hmm.

I'm actually f quite sure that the feeding the features into the system and training it up,

What if?

that that I think that's this that's essentially the same as we use with the ce with the P L P fe features. And that's obviously working great. So. I um.

Yeah. There could be a bug in in the somewhere before that.

There we could the another degree of freedom is how do you generate the K L T transform?

Mm - hmm.

Right? We to

That's.

Right.

well, and another one is the normalization of the inputs to the net.

Yeah.

These nets are trained with particular normalization and when that gets screwed up it it can really hurt it.

I'm doing what Eric E Eric coached me through then that part of it, so I'm pretty confident in that.

OK.

I mean, the only slight difference is that I use normalization values that, um, Andreas calculated from the original  PLP,

Right.

which is right.

Right.

N Yeah. So, I u I do Oh, we actually don't do that normalization for the PLP, do we? For the st just the straight PLP features?

No. The the SRI system does it.

S R I system does that. Right.

Yeah.

Right. Well, you might e e

So, there's there is there is room for bugs that we might not have discovered,

So that's that's another Yeah.

Yeah.

Mm - hmm.

Yeah. I I would actually double check with Stephane at this point,

but.

cuz he's probably the one here I mean, he and Dan are the ones who are at this point most experienced with the tandem

Mm - hmm.

thing and there may there may be some little bit here and there that is not not being handled right.

Yeah. It's hard with features, cuz you don't know what they should look like. I mean, you can't just, like, print the the values out in ASCII and, you know, look at them, see if they're.

Not unless you had a lot of time

Well.

and.

eh, and also they're not I mean, as I understand it, you you don't have a way to optimize the features for the final word error. Right?

Right.

I mean, these are just discriminative, but they're not, um, optimized for the final.

They're optimized for phone discrimination, not for.

Right. So it there's always this question of whether you might do better with those features if there was a way to train it for the word error metric that you're actually that you're actually.

That's right. Well, the other Yeah, th the.

Mm - Mmm.

Well, you actually are. But but it but in an indirect way.

Well, right. It's indirect, so you don't know.

So wha w what an and you may not be in this case, come to think of it, because, uh, you're just taking something that's trained up elsewhere. So, what what you what you do in the full procedure is you, um, uh, have an embedded training. So in fact you the the net is trained on, uh, uh, a, uh, Viterbi alignment of the training data that comes from your full system. And so that's where the feedback comes all around, so that it is actually discriminant. You can prove that it's it's a, uh If you believe in the Viterbi assumption that, uh, getting the best path, uh, is almost equivalent to getting the best, uh, total probability, um, then you actually do improve that by, uh by training up on local local, uh local frames. But, um, we aren't actually doing that here, because we did we did that for a hybrid system, and now we're plugging it into another system and so it isn't i i i it wouldn't quite apply here.

Do y

So another huge experiment we could do would be to take the tandem features, uh, do SRI forced alignments using those features, and then re - do the net with those.

Mm - hmm.

Mmm, uh Exactly. Exactly.

Yeah.

So that you can optimize it for the word error.

But.

Yeah. Another thing is since you're not using the net for recognition per se but just for this transformation, it's probably bigger than it needs to be.

Yeah.

So that would save a lot of time.

And there's a mismatch in the phone sets. So, you're using a l a long a larger phone set than what.

Mmm.

Yeah. Actually all those things could could could could, uh could affect it as well.

Yeah. Yeah.

The other thing, uh, just to mention that Stephane this was an innovation of Stephane's, which was a pretty neat one, uh, and might particularly apply here, given all these things we're mentioning. Um, Stephane's idea was that, um, discriminant, uh, approaches are great. Even the local ones, given, you know, these potential outer loops which, you know, you can convince yourself turn into the global ones. Um, however, there's times when it is not good. Uh, when something about the test set is different enough from the training set that that, uh, the discrimination that you're learning is is is not a good one.

Mm - hmm.

So, uh, his idea was to take as the input feature vector to the, uh, Gaussian mixture system, uh, a concatenation of the neural net outputs and the regular features.

Oh, we already talked about that.

Yeah. That.

Yeah.

El

Mm - hmm.

Didn't you did you do that already

Yeah. No, but we we when when we when I first started corresponding with Dan about how to go about this, I think that was one of the things that we definitely went there.

or? Oh. That makes a lot of sense. Huh.

Yeah. Yeah. I mean, I'm sure that Stephane wasn't the first to think of it,

Yeah.

but actually Stephane did it

Uh - huh. And i does it help?

and and and it helped a lot.

Oh, OK.

Yeah. So that's that that's our current best best system in the, uh uh, in the Aurora thing.

Oh. OK.

Yeah. That makes sense.

And do you do a KLT transform on the con on the combined feature vector?

Yeah.

As you should never do worse.

I I, uh, missed what you said.

Do you d you do a KLT transform on the combined feature vector?

Yeah.

OK.

Well, actually, I, uh you should check with him, because he tried several different combinations.

Because you end up with this huge feature vector, so that might be a problem, a unless you do some form of dimensionality reduction.

Yeah. I, uh, th what I don't remember is which came out best. So he did one where he put o put e the whole thing into one KLT, and another one, since the the PLP things are already orthogonalized, he left them alone and and just did a KLT on the on the on the net outputs

Mm - hmm. Mmm.

and then concatenated that. And I don't remember which was better.

Did he did he try to? So he always ended up with a feature vector that was twice as long as either one of the?

No. I don't know, i I I don't know. You have to check with him.

Yeah.

OK. Actually, I have to run.

I'm into big ideas these days.

Yeah.

Uh.

We need to close up cuz I need to save the data and, um, get a call.

Not to mention the fact that we're missing snacks. Yeah.

Right.

Uh

Did people wanna do the digits

Um.

or, um, do them together?

I I g I think, given that we're in a hurry for snacks, maybe we should do them together.

I don't know. Should we just? OK. I mean, are we trying to do them in synchrony? That might be fun.

Well, it's it's it's not You know, it's not gonna work out

Adam's not here, so he's not here to tell me no.

but we could we could just, uh, uh, see if we find a rhythm, you know, what.

Sure.

Uh, O's or zeroes, we wanna agree on that?

Maybe just whatever people would naturally do? I don't know.

Oh, but if we were a singing group, we would wanna decide. Right?

Be harmony. Yeah.  Yeah.

Mine's identical to yours.

We might wa

Is that correct?

Sorry. So I set up and we didn't have enough digit forms

Oh. I see.

So these are excellent.

so I xeroxed the same one seven times.

Oh. I see.

Why don't we do zer i Anyone have a problem with saying zero? Is zero OK?

No.

Yeah.

OK. One and a two and three.

e

Once more with feeling.

And th

No, just k just kidding. Oh, yeah. It was.

Question: What did PhD G say about SRI recognizer in the meeting?

Response: PhD G did not say anything regarding  SRI recognizer in the meeting.

Question: Besides the SRI recognizer, was there any other topic PhD G brought up?

Response: The meeting does not contain any suggestions made by PhD G on the topic SRI recogniser, hence the answer to this question is not present in the meeting.

Question: Were there any points of discussion that were left unresolved during the meeting?

Response:
There were a few points of discussion that were left unresolved during the meeting:
* The issue of recognition accuracy on automatically segmented meetings was brought up, but a clear solution was not reached.
* The participants agreed that they would need to obtain a better estimate of the turnaround time for Meeting Recorder to transcribe the meetings before they could determine how many transcribers to hire.
* PhD G mentioned that they were still experimenting with different segmentation strategies to improve system performance.

Question: What was their conclusion about the turnaround time for Meeting Recorder to transcribe the meetings?

Response: The participants agreed that they would need to obtain a better estimate of the turnaround time for Meeting Recorder to transcribe the meetings before they could determine how many transcribers to hire.

Question: Can you summarize the discussion about the IBM transcription service?

Response:
Here are some points discussed in the meeting regarding the IBM transcription service:
* Brian, from IBM, discussed an alternative to the single beep at the end of each utterance.
* The transcriber preferred a beep at the beginning of each number, followed by the number, and another beep at the end.
* This new approach would help the transcribers avoid getting lost during the transcription process.
* PhD D recorded the numbers one through ten three times at three different speeds, and Brian chose the fastest one.
* Brian and his team are working on generating these chunked files and will send them to the participants for testing.

Question: Why did Professor F suggest a beep at the beginning of each number followed by another beep at the end, as an alternative to the single beep at the end of each utterance?

Response: Professor F did not suggest a beep at the beginning of each number followed by another beep at the end, as an alternative to the single beep at the end of each utterance.

Question: Why did Professor F propose that the transcribers insert a beep at the start and end of every number during transcription?

Response: The meeting transcript does not provide any information regarding any proposal by Professor F that transcribers insert a beep at the start and end of every number during transcription.

Question: Why did Professor F think the transcriber would prefer a beep at the beginning of each number followed by another beep at the end, instead of a single beep at the end of each utterance?

Response: Professor F did not think the transcriber would prefer a beep at the beginning of each number followed by another beep at the end, instead of a single beep at the end of each utterance.

Question: Why, according to Professor F, did the transcriber prefer a beep at the beginning of each number followed by another beep at the end, as opposed to a single beep at the end of each utterance?

Response: The reason why Professor F suggested that the transcriber would prefer a beep at the beginning of each number followed by another beep at the end instead of a single beep at the end of each utterance is not explicitly mentioned in the meeting.

Question: Why did Professor F think the transcriber would prefer beep-number-beep over a single beep at the end of each utterance?

Response: Professor F did not think that the transcriber would prefer a beep at the beginning of each number followed by another beep at the end, instead of a single beep at the end of each utterance.