Debugger permissions?

Mac OSX specific forum
ProphetOfDoom
User
User
Posts: 84
Joined: Mon Jun 30, 2008 4:36 pm
Location: UK

Debugger permissions?

Post by ProphetOfDoom »

Hullo there friends,
I want to write a debugger for macOS Catalina. However I think Apple’s security “features” might cause problems. Here is a list of the steps needed just to get the GNU Debugger working:

https://www.google.com/amp/s/timnash.co ... cos/%3Famp

I’m surprised the author managed to write the tutorial without swearing! It’s ridiculous.
So my two questions are:
1) How do I get the right privileges for a debugger I’ve written myself in Purebasic? Do I just follow the same steps as for GDB?
2) When I come to actually ship my app, how can I make sure the end user doesn’t have to go through such a miserable process just to get it working?! I didn’t need to do anything like this when I installed PB... is that because PB is whitelisted like the binary is known to Apple? I’d like to know what’s going on?
Thank you!
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

This article is not relevant, because it was written only to avoid the problems homebrew gdb port for macos had, 'brew install gdb' totally works now, but you have to sign the binary anyway (homebrew tells you to do that):

Code: Select all

==> Caveats
gdb requires special privileges to access Mach ports.
You will need to codesign the binary. For instructions, see:

  https://sourceware.org/gdb/wiki/BuildingOnDarwin

On 10.12 (Sierra) or later with SIP, you need to run this:

  echo "set startup-with-shell off" >> ~/.gdbinit
So, answering your questions:
1. Yes, you need to obtain a developer certificate and pre-sign your debugger with it. The end user won't need to sign anything on their end.
2. Yes, you can package your software to dmg or pkg and distribute it as usual.

I personally use standard codesign from xcode and node-appdmg for dmg creation. You can check out my whole build script here.
ProphetOfDoom
User
User
Posts: 84
Joined: Mon Jun 30, 2008 4:36 pm
Location: UK

Re: Debugger permissions?

Post by ProphetOfDoom »

Thanks deseven for confirming what I thought... things are clearer now. I will download the newest GDB from Homebrew to try it when I get a moment.
When I was Googling for information on this, I found an article by the developers of IDA Pro (a premium debugging/reversing app) and it said their debugging server uses interprocess communication rather than ptrace/mach ports... due to Apple crippling everything. It didn't say in more detail how it works (shared memory or pipes maybe). I wonder if PureBasic uses similar techniques.
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

Signing is a thing on both Windows and macOS, you can't use low level functions without it, like you can't, for example, install unsigned drivers on Windows. User-level software does not require signing in general, although on both macOS and Windows you'll get a warning on a first run. If you're not okay with that and you don't want to spend money on signing ($100 per year on macOS, $500+ on Windows), you will obviously have to provide user instructions on how to run and use your software, depending on what capabilities you need.

PureBasic uses the debugger that is a part of the debugged executable, it doesn't need any special capabilities and it doesn't require signing of any sort (although PB is signed anyway).
ProphetOfDoom
User
User
Posts: 84
Joined: Mon Jun 30, 2008 4:36 pm
Location: UK

Re: Debugger permissions?

Post by ProphetOfDoom »

Hi there again. Thanks for the reply.
I think because I only used Linux for approx 10 years, it’s a bit of a culture shock to come back to proprietary OSs and find that it’s no longer the Wild West (in terms of security) like it used to be.
Back in about 2010 I wrote a little debugger for Microsoft Windows (I never completed it though). I was too intimidated by the COFF specification (generating raw sections etc) so I went my own route as regards communication of the target program with the debugger. After every source line change, the target program would send a “location update” to the debugger via IPC. I remember very clearly that everything was a horrible mess of named pipes and threads. Oh my, I hate multi threading.
If PureBasic does a similar thing that might make the technique seem more respectable! Maybe I should go that route again. It’s duplicating work though as I already have DWARF implemented in my BASIC compiler and can debug fine with lldb. I just want to make my own GUI debugger.
I suppose I should actually write the debugger before I worry about signing.
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

Oh, i have to mention then that threads in PB don't work well on macOS. I tried several different approaches before, but all of them led to random crashes, even when there are no shared resources and gui operations. Some integrated libs (i can pinpoint at least libcurl) are totally incompatible with threading on macOS. And threadsafe flag does absolutely nothing.

PB support for macOS in general leaves a lot to be desired, especially if you're writing something even remotely complex. Many core libraries are either broken or have a lot of quirks you need to know about and that don't exist on other OSes. I ended up rewriting many functions under native API just to avoid several bugs. Fortunately, there are several active mac PB users who are usually trying to help.

So yeah, i'm afraid signing will be the least of your problems. Good luck!
Wolfram
Enthusiast
Enthusiast
Posts: 567
Joined: Thu May 30, 2013 4:39 pm

Re: Debugger permissions?

Post by Wolfram »

Oh, i have to mention then that threads in PB don't work well on macOS.
I work always with threads and have no problems.
Be sure you have on the thread save option in the compiler option. Check your code for memory leaks and run it with purifier.
Use alway PosteEvent() to change GUI stuff and Mutex if you work on one memory from different threads.
macOS Catalina 10.15.7
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

Wolfram wrote:Be sure you have on the thread save option in the compiler option. Check your code for memory leaks and run it with purifier.
Use alway PosteEvent() to change GUI stuff and Mutex if you work on one memory from different threads.
Please re-read my previous post.
ProphetOfDoom
User
User
Posts: 84
Joined: Mon Jun 30, 2008 4:36 pm
Location: UK

Re: Debugger permissions?

Post by ProphetOfDoom »

Hmm okay haha, that’s slightly alarming as I’m quite good at crashing stable thread implementations... so one that already tends to crash... doesn’t sound good!
I really wanted to use PureBasic because it just seems so simple compared to objective c, and I would only have to write the code once for all OSs. I think I’ll give it a try anyway. Maybe just use Mach ports/ptrace/DWARF and accept that my macOS is going to treat me as an intruder...
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Debugger permissions?

Post by mk-soft »

I did not notice here under macOS that threads do not work.
As deseven has already described, you have to pay attention to the rules as with all OS.

There is a special rule under macOS for the use of CocoaMessage in threads. Here you have to create an own cocoa pool in threads.
Small example to made own function thread safe.

Code: Select all

;-TOP

#NSApplicationActivateAllWindows = 1 << 0
#NSApplicationActivateIgnoringOtherApps = 1 << 1

Threaded _IsMainScope
_IsMainScope = #True

Procedure ShowNamedWindow(Name.s) ;  ; Result = Count of Windows
  Protected RunningApps.i, RunningAppsCount.i, RunningApp.i, AppName.s, i, cnt, Pool
  
  If Not _IsMainScope
    Pool = CocoaMessage(0, 0, "NSAutoreleasePool new")
  EndIf
  
  RunningApps = CocoaMessage(0, CocoaMessage(0, 0, "NSWorkspace sharedWorkspace"), "runningApplications")
  RunningAppsCount = CocoaMessage(0, RunningApps, "count")
  
  i = 0
  While i < RunningAppsCount
    RunningApp = CocoaMessage(0, RunningApps, "objectAtIndex:", i)
    AppName.s = PeekS(CocoaMessage(0, CocoaMessage(0, RunningApp, "localizedName"), "UTF8String"), -1, #PB_UTF8)
    If Name = AppName
      CocoaMessage(0, RunningApp, "activateWithOptions:", #NSApplicationActivateIgnoringOtherApps)
      cnt + 1
    EndIf
    i + 1
  Wend
  
  If Pool
    CocoaMessage(0, Pool, "release")
  EndIf
  
  ProcedureReturn cnt
EndProcedure

ShowNamedWindow("Safari")
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

Okay, let's just say i can't code since nobody else is having any problems with threads in PB for macOS, i'm totally fine with that :)
Sorry for misinformation!
User avatar
Shardik
Addict
Addict
Posts: 1988
Joined: Thu Apr 21, 2005 2:38 pm
Location: Germany

Re: Debugger permissions?

Post by Shardik »

deseven wrote:Okay, let's just say i can't code since nobody else is having any problems with threads in PB for macOS, i'm totally fine with that :)
Sorry for misinformation!
Nobody claimed that you can't code! We all know from your postings and your GitHub examples that you are an experienced MacOS programmer and I take your problems with random crashes in threaded code very seriously. I also didn't know mk-soft's workaround of using an AutoreleasePool. Did you try it already in one of your failing thread examples?

Nevertheless it would be nice if you could post examples of code leading to random crashes.
User avatar
mk-soft
Always Here
Always Here
Posts: 5333
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Debugger permissions?

Post by mk-soft »

@deseven

I didn't mean it that way either. The solution also does not come directly from me. I also had crashes with threads and remembered that in some examples the one with the extra pool, but in another context in the MainScope to save objects after a WaitWindowEvent, as well as in examples written directly in Objective-C.

Unfortunately there are also OS specific external libraries that are not Threadsafe. These are also available in Window and Linux. Fortunately they are very rare.

So I take back, that I have no problems with threads 8) :wink:
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

No guys, that wasn't sarcasm or anything, i honestly may just not know something and it all happened about 2 to 3 years ago when i gave up. I mean i totally understand that it must be something wrong with my code if other people are having little to no problems.

It was a project that periodically polled remote server with http requests, and it was getting a huge json in return that was needed to be parsed. There were no simultaneously used shared resources (and no mutexes because of that), no GUI interaction from threads, only a couple of PostEvent and one single GetGadgetState (removing it did nothing). And it was crashing, totally randomly, sometimes in a minute, sometimes in a day. Threadsafe flag didn't do anything useful, it only replaced crashes with freezes.

I spent countless hours experimenting and rewriting it, trying to find the cause, but nothing worked. PB debugger was pointing on completely random lines so i started stripping the program down piece by piece and in the end i was left with either libcurl or JSON libraries. Unfortunately there never was a short example, because at this point i decided that it just not worth it and rewrote everything from scratch on Swift. Never touched threads in macOS PB since then.

And no, i never tried this NSAutoreleasePool workaround, never heard about it before.
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Re: Debugger permissions?

Post by deseven »

Well, i decided to give it a try one more time and ended up with the same result. At some point i started getting "Invalid memory access" errors pointing to WaitWindowEvent(). Sometimes there is no error, the app just hangs indefinitely. Sometimes it works fine for hours and hours.

Image

I know that symptoms may indicate that there is a race condition of some sort, so i added mutexes just in case. A lot of mutexes. I made it so that i basically have a multithreaded app where only one thread (including the main one) works at the same time. Nothing has changed.

I believe i made no obvious mistakes:
- nothing touches GUI in any thread
- threadsafe flag is enabled
- no resources that can be used in multiple threads at the same time
- no globals, EnableExplicit, pretty straightforward procedures that only do one specific thing

By the way, this code also uses http and json libraries, just like the one i previously had problems with :)

I still don't know if it's something wrong with me, my code, or PB. I tried to strip everything down to a simple example, but it looks like bad things start to happen only when the app reaches some level of complexity?.. Yeah, i know how it sounds and i have no explanation other than maybe there is some sort of an internal buffer that overflows with events or something.

Anyway, i wasted 2 weeks on this project and now it's time to rewrite everything in Swift. I'll leave the code in "purebasic" branch in case someone want to dig into that in the future:
https://github.com/deseven/iCanHazMusic/tree/purebasic
Post Reply