I am runnning mambo and joomla on the same domain... one to handle regular members and one for subscription.. Below is what I used to get it to work and it does great..
You just have to make sure you turn off SEF.. of course any pages that you created using SEF pages will have to be rewritten.. But now you can go to mydomain.com/yourname for the profile.
##
# @version $Id: htaccess.txt,v 1.1 2005/07/22 01:51:57 eddieajau Exp $
# @package Mambo
# @copyright (C) 2000 - 2005 Miro International Pty Ltd
# @license
www.gnu.org/copyleft/gpl.html GNU/GPL
# Mambo is Free Software
##
#
# mod_rewrite in use
#
RewriteEngine On
# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update YourMamboDirectory (just / for root)
# RewriteBase /YourMamboDirectory
#
# Rules
#
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)/?$ /myscifi/index.php?option=com_comprofiler&task=userProfile&user='$1' [L]