[GRASS-dev] nviz python thoughts

I was thinking that all Python stuff (yes, I know it's experimental) that compiles and runs on OSX should be so without fuss for the 6.4 release. In particular NVIZ. Currently, to get libnviz to compile we must disable/delete render.c, which makes the not-working-yet-on-OSX nviz_cmd fail to compile.

If only nviz_cmd needs the render.c functions, could render.c be moved to the nviz_cmd folder? This would let libnviz compile on OSX and make the wx GUI nviz operational. nviz_cmd would still fail to compile as expected, without affecting the GUI nviz compilation.

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"Time is an illusion - lunchtime doubly so."

- Ford Prefect

Hi,

2008/12/20 William Kyngesburye <woklist@kyngchaos.com>:

I was thinking that all Python stuff (yes, I know it's experimental) that
compiles and runs on OSX should be so without fuss for the 6.4 release. In
particular NVIZ. Currently, to get libnviz to compile we must
disable/delete render.c, which makes the not-working-yet-on-OSX nviz_cmd
fail to compile.

If only nviz_cmd needs the render.c functions, could render.c be moved to
the nviz_cmd folder? This would let libnviz compile on OSX and make the wx
GUI nviz operational. nviz_cmd would still fail to compile as expected,
without affecting the GUI nviz compilation.

sure, possible (see the attached patch). Anyway it would be better to
get it compiled and functional within nvizlib. Sorry I have no Mac for
testing...

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

(attachments)

nviz_cmd_render.diff.gz (1.69 KB)

William Kyngesburye wrote:

I was thinking that all Python stuff (yes, I know it's experimental)
that compiles and runs on OSX should be so without fuss for the 6.4
release. In particular NVIZ. Currently, to get libnviz to compile we
must disable/delete render.c, which makes the not-working-yet-on-OSX
nviz_cmd fail to compile.

If only nviz_cmd needs the render.c functions, could render.c be moved
to the nviz_cmd folder? This would let libnviz compile on OSX and
make the wx GUI nviz operational. nviz_cmd would still fail to
compile as expected, without affecting the GUI nviz compilation.

Only nviz_cmd needs it at the moment, but the code is sufficiently
general that it belongs in the library.

Remind me: what is the specific problem with compiling it on OSX?

--
Glynn Clements <glynn@gclements.plus.com>

On Dec 20, 2008, at 11:08 AM, Glynn Clements wrote:

William Kyngesburye wrote:

I was thinking that all Python stuff (yes, I know it's experimental)
that compiles and runs on OSX should be so without fuss for the 6.4
release. In particular NVIZ. Currently, to get libnviz to compile we
must disable/delete render.c, which makes the not-working-yet-on-OSX
nviz_cmd fail to compile.

If only nviz_cmd needs the render.c functions, could render.c be moved
to the nviz_cmd folder? This would let libnviz compile on OSX and
make the wx GUI nviz operational. nviz_cmd would still fail to
compile as expected, without affecting the GUI nviz compilation.

Only nviz_cmd needs it at the moment, but the code is sufficiently
general that it belongs in the library.

Remind me: what is the specific problem with compiling it on OSX?

Opening and drawing into a Mac OpenGL window. In all the other past and present forms, we've had some generic window system to use (Togl, Tk, wxPython), but it looks like this one is more low-level.

I wonder if it can be made to require and use wxPython, just like the GUI version of nviz (but without the rest of the GUI)?

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"We are at war with them. Neither in hatred nor revenge and with no particular pleasure I shall kill every ___ I can until the war is over. That is my duty."

"Don't you even hate 'em?"

"What good would it do if I did? If all the many millions of people of the allied nations devoted an entire year exclusively to hating the ____ it wouldn't kill one ___ nor shorten the war one day."

<Ha, ha> "And it might give 'em all stomach ulcers."

- Tarzan, on war

William Kyngesburye wrote:

>> I was thinking that all Python stuff (yes, I know it's experimental)
>> that compiles and runs on OSX should be so without fuss for the 6.4
>> release. In particular NVIZ. Currently, to get libnviz to compile
>> we
>> must disable/delete render.c, which makes the not-working-yet-on-OSX
>> nviz_cmd fail to compile.
>>
>> If only nviz_cmd needs the render.c functions, could render.c be
>> moved
>> to the nviz_cmd folder? This would let libnviz compile on OSX and
>> make the wx GUI nviz operational. nviz_cmd would still fail to
>> compile as expected, without affecting the GUI nviz compilation.
>
> Only nviz_cmd needs it at the moment, but the code is sufficiently
> general that it belongs in the library.
>
> Remind me: what is the specific problem with compiling it on OSX?

Opening and drawing into a Mac OpenGL window. In all the other past
and present forms, we've had some generic window system to use (Togl,
Tk, wxPython), but it looks like this one is more low-level.

I wonder if it can be made to require and use wxPython, just like the
GUI version of nviz (but without the rest of the GUI)?

1. It's not trying to draw into a window, but a pixmap or similar
off-screen surface, so that the image can be saved into a file,
without any of the crud (tooltips, mouse cursor, etc) which you might
get from drawing to a window then grabbing it.

2. We don't need to get it to run, just compile. We just need to
ensure that the functions which are called actually exist and that
values have the correct types. If you can't figure out what the
correct syntax is, just remove (or put "#if 0 ..." around) any parts
that cause compilation errors. Removing the entire file is overkill.

3. The last record I can find of this discussion was back in July:

http://www.mail-archive.com/grass-dev@lists.osgeo.org/msg04540.html

Some suggestions were made, but I'm not sure if anyone actually tried
them.

4. I still don't know what the *actual* error is.

--
Glynn Clements <glynn@gclements.plus.com>

On Dec 20, 2008, at 10:33 PM, Glynn Clements wrote:

1. It's not trying to draw into a window, but a pixmap or similar
off-screen surface, so that the image can be saved into a file,
without any of the crud (tooltips, mouse cursor, etc) which you might
get from drawing to a window then grabbing it.

Ah, so it's not really a replacement for the togl-based interactive nviz. But still renders to a window? After poking around with stuff below, I wonder if this can work on OSX - I get the impression that a window must be attached to a Mac application somehow, not just a faceless CLI program.

3. The last record I can find of this discussion was back in July:

http://www.mail-archive.com/grass-dev@lists.osgeo.org/msg04540.html

Some suggestions were made, but I'm not sure if anyone actually tried
them.

There is some information on AGL at:

[1] http://www.mesa3d.org/brianp/sig97/agl.txt
[2] http://www.mesa3d.org/brianp/sig97/compare.htm

but I don't know how much of that refers to "classic" rather than OSX.

1997 = pre-OSX. The Mac window "convenience routines" mentioned are what we need, but are not available there and would be Classic-based anyways.

[3] http://www.iua.upf.es/~ggeiger/redbookhtml/apc.html

That mentions QuickDraw, which is from Classic.

Though, the Classic stuff in QuickDraw is available via Carbon API. This is deprecated in OSX 10.4, but still available. Carbon is not 64bit-enabled in OSX 10.5. The replacement to QuickDraw is Quartz 2D, available starting with OSX 10.4. So we may need to write 2 different versions of this. I'd say start with Quartz 2D, then figure out QuickDraw later...

... oh, AGL seems to be Carbon-based. Here's something about AGL in Leopard which has some references to non-carbon alternatives:

http://developer.apple.com/qa/qa2007/qa1523.html

... Some sample code from Apple - it's about using surface textures, but the window parts could be useful:

http://developer.apple.com/samplecode/AGLSurfaceTexture/index.html#//apple_ref/doc/uid/DTS10000508

There is a pbuffer example also:

http://developer.apple.com/samplecode/GLCarbon1ContextPbuffer/index.html#//apple_ref/doc/uid/DTS10003144

4. I still don't know what the *actual* error is.

I forgot I could make a subdir :wink:

render.c:204: error: ‘mac_win’ undeclared (first use in this function)
render.c:204: error: (Each undeclared identifier is reported only once
render.c:204: error: for each function it appears in.)

Looks like mac_win is just a placeholder here. For now, comment this line and have Nviz_make_current_render_window return 0 for OPENGL_AQUA?

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

The equator is so long, it could encircle the earth completely once.

William Kyngesburye wrote:

> 1. It's not trying to draw into a window, but a pixmap or similar
> off-screen surface, so that the image can be saved into a file,
> without any of the crud (tooltips, mouse cursor, etc) which you might
> get from drawing to a window then grabbing it.
>
Ah, so it's not really a replacement for the togl-based interactive
nviz. But still renders to a window? After poking around with stuff
below, I wonder if this can work on OSX - I get the impression that a
window must be attached to a Mac application somehow, not just a
faceless CLI program.

lib/nviz/render.c is a replacement for nviz/src/do_zoom.c. It creates
an off-screen context, e.g. for when you want to save the output to an
image file. It could be used as part of an interactive application
(e.g. nviz2), or as part of a command-line application (e.g.
nviz_cmd).

... oh, AGL seems to be Carbon-based. Here's something about AGL in
Leopard which has some references to non-carbon alternatives:

http://developer.apple.com/qa/qa2007/qa1523.html

That could be useful for actually getting it to work, but for now we
just need to get render.c to compile on OSX. It doesn't actually need
to work for normal use of nviz2, it just needs to not prevent lib/nviz
from compiling.

... Some sample code from Apple - it's about using surface textures,
but the window parts could be useful:

Using OpenGL in a window should be taken care of by wxGLCanvas. It's
getting off-screen contexts which we (probably) have to do ourselves.

The situation is the same with the existing nviz; Togl provides a
canvas on which OpenGL can be used. The only glX/wgl/agl functions
outside of Togl are the code in do_zoom.c to get an off-screen context
for saving output to files.

> 4. I still don't know what the *actual* error is.
>
I forgot I could make a subdir :wink:

render.c:204: error: �mac_win� undeclared (first use in this function)
render.c:204: error: (Each undeclared identifier is reported only once
render.c:204: error: for each function it appears in.)

Looks like mac_win is just a placeholder here. For now, comment this
line and have Nviz_make_current_render_window return 0 for OPENGL_AQUA?

In that case, just:

     /* TODO: mac_win */
- aglMakeCurrent((AGLDrawable) mac_win, rwin->contextId);
+ aglMakeCurrent((AGLDrawable) 0, rwin->contextId);

should get it to compile (which is all we're after for now).

--
Glynn Clements <glynn@gclements.plus.com>

On Dec 21, 2008, at 12:14 PM, Glynn Clements wrote:

... Some sample code from Apple - it's about using surface textures,
but the window parts could be useful:

Using OpenGL in a window should be taken care of by wxGLCanvas. It's
getting off-screen contexts which we (probably) have to do ourselves.

The situation is the same with the existing nviz; Togl provides a
canvas on which OpenGL can be used. The only glX/wgl/agl functions
outside of Togl are the code in do_zoom.c to get an off-screen context
for saving output to files.

I guess I'm missing something here. I understand about the offscreen part, so where does the window come in in render.c? Is it a hidden window, just so we can get the context?

Looks like mac_win is just a placeholder here. For now, comment this
line and have Nviz_make_current_render_window return 0 for OPENGL_AQUA?

In that case, just:

    /* TODO: mac_win */
- aglMakeCurrent((AGLDrawable) mac_win, rwin->contextId);
+ aglMakeCurrent((AGLDrawable) 0, rwin->contextId);

should get it to compile (which is all we're after for now).

Should it return 0, just to be nice? presumably, nviz_cmd would fail gracefully if the render.c functions return a failure, but would it do so on success but with a nonexistent context?

Anyways, the online API docs that we found way back when to get the function names seems to have been wrong. Strangely, render.c compiles with the wrong names, but doesn't link:

Undefined symbols:
   "_aglChoosePixelFmt", referenced from:
       _Nviz_create_render_window in render.o
   "_aglCreateAGLPixmap", referenced from:
       _Nviz_create_render_window in render.o
   "_aglMakeCurrent", referenced from:
       _Nviz_make_current_render_window in render.o
   "_aglDestroyAGLPixmap", referenced from:
       _Nviz_destroy_render_window in render.o

Indeed, these are not in the AGL headers. What I can find is:

extern AGLPixelFormat aglChoosePixelFormat(const AGLDevice *gdevs, GLint ndev, const GLint *attribs);

extern GLboolean aglSetCurrentContext(AGLContext ctx);
(I didn't see any form of "make current", with a context AND display/window parameters)

There don't appear to be any pixmap functions, but there are pbuffer functions.

Looks like nviz.h has the right names for types in the render_window struct.

PS. render.h is not included by render.c, or anything else in GRASS. It looks like everything in render.h is in nviz.h. Maybe render.h should be deleted? (it was confusing me with it's duplicated render_window declaration)

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"We are at war with them. Neither in hatred nor revenge and with no particular pleasure I shall kill every ___ I can until the war is over. That is my duty."

"Don't you even hate 'em?"

"What good would it do if I did? If all the many millions of people of the allied nations devoted an entire year exclusively to hating the ____ it wouldn't kill one ___ nor shorten the war one day."

<Ha, ha> "And it might give 'em all stomach ulcers."

- Tarzan, on war

Hi,

2008/12/21 William Kyngesburye <woklist@kyngchaos.com>:

PS. render.h is not included by render.c, or anything else in GRASS. It
looks like everything in render.h is in nviz.h. Maybe render.h should be
deleted? (it was confusing me with it's duplicated render_window
declaration)

done in r34980 (devbr6) r34981 (trunk), r34982 (relbr64).

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

On Dec 21, 2008, at 12:51 PM, William Kyngesburye wrote:

Anyways, the online API docs that we found way back when to get the function names seems to have been wrong. Strangely, render.c compiles with the wrong names, but doesn't link:

Undefined symbols:
"_aglChoosePixelFmt", referenced from:
     _Nviz_create_render_window in render.o
"_aglCreateAGLPixmap", referenced from:
     _Nviz_create_render_window in render.o
"_aglMakeCurrent", referenced from:
     _Nviz_make_current_render_window in render.o
"_aglDestroyAGLPixmap", referenced from:
     _Nviz_destroy_render_window in render.o

Indeed, these are not in the AGL headers. What I can find is:

extern AGLPixelFormat aglChoosePixelFormat(const AGLDevice *gdevs, GLint ndev, const GLint *attribs);

extern GLboolean aglSetCurrentContext(AGLContext ctx);
(I didn't see any form of "make current", with a context AND display/window parameters)

There don't appear to be any pixmap functions, but there are pbuffer functions.

Looks like nviz.h has the right names for types in the render_window struct.

I'm starting to make some sense of the AGL context stuff. Here's a patch that lets render.c compile and link, though it's still non-functional:

(attachments)

render.c.patch (1.51 KB)

William Kyngesburye wrote:

>> ... Some sample code from Apple - it's about using surface textures,
>> but the window parts could be useful:
>
> Using OpenGL in a window should be taken care of by wxGLCanvas. It's
> getting off-screen contexts which we (probably) have to do ourselves.
>
> The situation is the same with the existing nviz; Togl provides a
> canvas on which OpenGL can be used. The only glX/wgl/agl functions
> outside of Togl are the code in do_zoom.c to get an off-screen context
> for saving output to files.

I guess I'm missing something here. I understand about the offscreen
part, so where does the window come in in render.c? Is it a hidden
window, just so we can get the context?

There's no inherent need for a window. In the X version, the windowId
field is actually a GLXPixmap. The only reference to an actual window
is to the root window (XCreatePixmap takes a window parameter which it
uses to determine the screen on which to create the pixmap).

>> Looks like mac_win is just a placeholder here. For now, comment this
>> line and have Nviz_make_current_render_window return 0 for
>> OPENGL_AQUA?
>
> In that case, just:
>
> /* TODO: mac_win */
> - aglMakeCurrent((AGLDrawable) mac_win, rwin->contextId);
> + aglMakeCurrent((AGLDrawable) 0, rwin->contextId);
>
> should get it to compile (which is all we're after for now).

Should it return 0, just to be nice? presumably, nviz_cmd would fail
gracefully if the render.c functions return a failure, but would it do
so on success but with a nonexistent context?

Who knows? nviz_cmd won't work unless you can figure out how to create
something to render into. Failing with a segfault probably isn't any
worse than failing to compile.

Anyways, the online API docs that we found way back when to get the
function names seems to have been wrong. Strangely, render.c compiles
with the wrong names, but doesn't link:

Undefined symbols:
   "_aglChoosePixelFmt", referenced from:
       _Nviz_create_render_window in render.o
   "_aglCreateAGLPixmap", referenced from:
       _Nviz_create_render_window in render.o
   "_aglMakeCurrent", referenced from:
       _Nviz_make_current_render_window in render.o
   "_aglDestroyAGLPixmap", referenced from:
       _Nviz_destroy_render_window in render.o

Indeed, these are not in the AGL headers. What I can find is:

extern AGLPixelFormat aglChoosePixelFormat(const AGLDevice *gdevs,
GLint ndev, const GLint *attribs);

extern GLboolean aglSetCurrentContext(AGLContext ctx);
(I didn't see any form of "make current", with a context AND display/
window parameters)

There don't appear to be any pixmap functions, but there are pbuffer
functions.

Looks like nviz.h has the right names for types in the render_window
struct.

Odd. I thought that it might have been a classic-vs-OSX issue, but
none of those names appear in togl.c (which supports both) either.

A bit more Googling produces results which include the minutes of an
OpenGL ARB meeting in 1995, Brian Paul's course from SIGGRAPH '97, and
references to the red book (which I have, dated 1997, and which lists
the non-existent functions in Appendix C).

Oh, I also found a reference which indicated that Maple used to ship
Mesa (with aglChoosePixelFmt) with the Mac version, and a CVS-web page
for Mesa where aglChoosePixelFormat() uses "aglChoosePixelFmt" in an
error message.

My guess is that the agl API was still provisional back when the red
book got printed, and that's what Mesa used, and between them they
comprised the de facto standard which persisted until Macs started
getting 3D hardware. At which point, it would have been necessary for
Apple to formalise the API, which may have required changes in order
to fully integrate with the windowing system.

--
Glynn Clements <glynn@gclements.plus.com>